A communications equipment manufacturing company has a product which is responsible for emitting informative signals. Company wants to build a machine learning model which can help the company to predict the equipment’s signal quality using various parameters.
The data set contains information on various signal tests performed:
The need is to build a regressor which can use these parameters to determine the signal strength or quality as number.
Liveplot.import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib.style as style; style.use('fivethirtyeight')
from IPython.display import clear_output,Markdown
%matplotlib inline
plt.rcParams["figure.figsize"] = (15,10)
pd.set_option('display.max_colwidth', 500)
from sklearn.metrics import roc_auc_score, mean_absolute_error, r2_score
from sklearn.preprocessing import StandardScaler,MinMaxScaler
from sklearn.model_selection import train_test_split
import sys
from itertools import permutations
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, BatchNormalization, Activation, Dropout
from tensorflow.keras.models import Model
from tensorflow.keras.initializers import RandomNormal,Constant
import optuna
df = pd.read_csv("Part- 1,2&3 - Signal.csv")
df.head()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 8 | Parameter 9 | Parameter 10 | Parameter 11 | Signal_Strength | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 7.4 | 0.70 | 0.00 | 1.9 | 0.076 | 11.0 | 34.0 | 0.9978 | 3.51 | 0.56 | 9.4 | 5 |
| 1 | 7.8 | 0.88 | 0.00 | 2.6 | 0.098 | 25.0 | 67.0 | 0.9968 | 3.20 | 0.68 | 9.8 | 5 |
| 2 | 7.8 | 0.76 | 0.04 | 2.3 | 0.092 | 15.0 | 54.0 | 0.9970 | 3.26 | 0.65 | 9.8 | 5 |
| 3 | 11.2 | 0.28 | 0.56 | 1.9 | 0.075 | 17.0 | 60.0 | 0.9980 | 3.16 | 0.58 | 9.8 | 6 |
| 4 | 7.4 | 0.70 | 0.00 | 1.9 | 0.076 | 11.0 | 34.0 | 0.9978 | 3.51 | 0.56 | 9.4 | 5 |
Parameter 1 to Parameter 11 are independent variables.Signal Strength is the independent variable.df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1599 entries, 0 to 1598 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Parameter 1 1599 non-null float64 1 Parameter 2 1599 non-null float64 2 Parameter 3 1599 non-null float64 3 Parameter 4 1599 non-null float64 4 Parameter 5 1599 non-null float64 5 Parameter 6 1599 non-null float64 6 Parameter 7 1599 non-null float64 7 Parameter 8 1599 non-null float64 8 Parameter 9 1599 non-null float64 9 Parameter 10 1599 non-null float64 10 Parameter 11 1599 non-null float64 11 Signal_Strength 1599 non-null int64 dtypes: float64(11), int64(1) memory usage: 150.0 KB
Parameter 1 to Parameter 11 are of data type float.Signal Strength is of datatype integerdtemp = pd.DataFrame(df.describe().T)
dtemp['dtypes'] = df.dtypes
dtemp['N-Uniques'] = df.nunique()
dtemp
| count | mean | std | min | 25% | 50% | 75% | max | dtypes | N-Uniques | |
|---|---|---|---|---|---|---|---|---|---|---|
| Parameter 1 | 1599.0 | 8.319637 | 1.741096 | 4.60000 | 7.1000 | 7.90000 | 9.200000 | 15.90000 | float64 | 96 |
| Parameter 2 | 1599.0 | 0.527821 | 0.179060 | 0.12000 | 0.3900 | 0.52000 | 0.640000 | 1.58000 | float64 | 143 |
| Parameter 3 | 1599.0 | 0.270976 | 0.194801 | 0.00000 | 0.0900 | 0.26000 | 0.420000 | 1.00000 | float64 | 80 |
| Parameter 4 | 1599.0 | 2.538806 | 1.409928 | 0.90000 | 1.9000 | 2.20000 | 2.600000 | 15.50000 | float64 | 91 |
| Parameter 5 | 1599.0 | 0.087467 | 0.047065 | 0.01200 | 0.0700 | 0.07900 | 0.090000 | 0.61100 | float64 | 153 |
| Parameter 6 | 1599.0 | 15.874922 | 10.460157 | 1.00000 | 7.0000 | 14.00000 | 21.000000 | 72.00000 | float64 | 60 |
| Parameter 7 | 1599.0 | 46.467792 | 32.895324 | 6.00000 | 22.0000 | 38.00000 | 62.000000 | 289.00000 | float64 | 144 |
| Parameter 8 | 1599.0 | 0.996747 | 0.001887 | 0.99007 | 0.9956 | 0.99675 | 0.997835 | 1.00369 | float64 | 436 |
| Parameter 9 | 1599.0 | 3.311113 | 0.154386 | 2.74000 | 3.2100 | 3.31000 | 3.400000 | 4.01000 | float64 | 89 |
| Parameter 10 | 1599.0 | 0.658149 | 0.169507 | 0.33000 | 0.5500 | 0.62000 | 0.730000 | 2.00000 | float64 | 96 |
| Parameter 11 | 1599.0 | 10.422983 | 1.065668 | 8.40000 | 9.5000 | 10.20000 | 11.100000 | 14.90000 | float64 | 65 |
| Signal_Strength | 1599.0 | 5.636023 | 0.807569 | 3.00000 | 5.0000 | 6.00000 | 6.000000 | 8.00000 | int64 | 6 |
Parameter 8 has a very short range of 0.001 and may not add significant variance to the overall data set.Parameter 5 has a very small interquartile range and with some significant outliers to the right of the curve.Signal Strength is a discrete integer between 3 to 8.df.isnull().sum() , df.isna().sum()
(Parameter 1 0 Parameter 2 0 Parameter 3 0 Parameter 4 0 Parameter 5 0 Parameter 6 0 Parameter 7 0 Parameter 8 0 Parameter 9 0 Parameter 10 0 Parameter 11 0 Signal_Strength 0 dtype: int64, Parameter 1 0 Parameter 2 0 Parameter 3 0 Parameter 4 0 Parameter 5 0 Parameter 6 0 Parameter 7 0 Parameter 8 0 Parameter 9 0 Parameter 10 0 Parameter 11 0 Signal_Strength 0 dtype: int64)
NAN or NULL in the given dataframe. No additional cleansing required.# Assign the target variable
target = 'Signal_Strength'
%run edafunc.py
cat_var_list,cont_var_list,cats_df1= create_variable_list(df,10,stats=True)
cats_df1
| Variable | Type | Categorical_Class | Uniques | N-Uniques | |
|---|---|---|---|---|---|
| 0 | Signal_Strength | Categorical | Multi | [5, 6, 7, 4, 8, 3] | 6 |
| 1 | Parameter 1 | Numeric | NaN | NaN | 96 |
| 2 | Parameter 2 | Numeric | NaN | NaN | 143 |
| 3 | Parameter 3 | Numeric | NaN | NaN | 80 |
| 4 | Parameter 4 | Numeric | NaN | NaN | 91 |
| 5 | Parameter 5 | Numeric | NaN | NaN | 153 |
| 6 | Parameter 6 | Numeric | NaN | NaN | 60 |
| 7 | Parameter 7 | Numeric | NaN | NaN | 144 |
| 8 | Parameter 8 | Numeric | NaN | NaN | 436 |
| 9 | Parameter 9 | Numeric | NaN | NaN | 89 |
| 10 | Parameter 10 | Numeric | NaN | NaN | 96 |
| 11 | Parameter 11 | Numeric | NaN | NaN | 65 |
sncp_c1 = sns.color_palette("muted", 40)
num_cols=4
display(Markdown("# <center><u>Univariate Analysis</u></center>\n\n"))
for color,index in enumerate((range(0,len(cont_var_list)))):
fig, ax = plt.subplots(1,num_cols,figsize=(20,4))
display(Markdown(f"## <center>`{cont_var_list[index].upper()}`</center></br>"))
sns.boxplot(x=df[cont_var_list[index]],color=sncp_c1[color],ax=ax[0]);
ax[0].set_title(f'Boxplot with Outliers \n',{'fontsize':15})
sns.histplot(x=df[cont_var_list[index]],kde=True,color=sncp_c1[color],ax=ax[1]);
ax[1].set_title(f'Histogram with Outliers \n',{'fontsize':15})
ol_temp = sp.outliers_iqr(df[cont_var_list[index]])
sns.boxplot(x=ol_temp,color=sncp_c1[color],ax=ax[2]);
ax[2].set_title(f'Boxplot Outliers removed\n',{'fontsize':15})
sns.histplot(x=ol_temp,color=sncp_c1[color],ax=ax[3],kde=True);
ax[3].set_title(f'Histogram Outliers removed\n',{'fontsize':15})
plt.show()
Parameter 5 and Parameter 8 have very short IQR.# Heatmap of of all variables
display(Markdown(f"## <center>`Heatmap of correlations between all features`</center></br>"))
fig,ax = plt.subplots(figsize=(20,10))
sns.heatmap(df.corr(),annot=True);
plt.title("\nHeatmap of correlations between all features\n",{'fontsize':20});
display(Markdown(f"## <center>`Pair Plots`</center></br>"))
sns.pairplot(data=df,diag_kind='kde',hue='Signal_Strength');
1.There are significant correlations between most of the variables.
2.Most significant correlations are between Parameter 1 and Parameter 3, Parameter 1 and Parameter 8, Parameter 6 and Parameter 7 at almost 0.67.
3.Signal Strength is highly correlated with Parameter 10 and Parameter 11
def show_variance_table(df,target,threshold=0.0):
from sklearn.feature_selection import VarianceThreshold
columns_to_show = df.drop(target,axis=1).columns
vthreshold = VarianceThreshold(threshold=threshold)
vthreshold.fit(df[columns_to_show].values)
vt_df = pd.DataFrame({'Feature':columns_to_show,
'Variance Threshold':vthreshold.variances_}
).sort_values(by='Variance Threshold',ascending=False)
return vt_df
vt_df = show_variance_table(df,target)
display(Markdown(f"## <center>`Variance Threshold Plot`</center></br>"))
sns.barplot(data= vt_df, x='Variance Threshold', y='Feature', orient='h');
plt.show();
display(vt_df)
| Feature | Variance Threshold | |
|---|---|---|
| 6 | Parameter 7 | 283.000000 |
| 5 | Parameter 6 | 71.000000 |
| 0 | Parameter 1 | 3.029521 |
| 3 | Parameter 4 | 1.986654 |
| 10 | Parameter 11 | 1.134937 |
| 2 | Parameter 3 | 0.037924 |
| 1 | Parameter 2 | 0.032042 |
| 9 | Parameter 10 | 0.028715 |
| 8 | Parameter 9 | 0.023820 |
| 4 | Parameter 5 | 0.002214 |
| 7 | Parameter 8 | 0.000004 |
Parameter 1, Parameter 4, Parameter 6, Parameter 7 and Parameter 11 are the top 5 variables showing high variance in the data.Parameter 8 shows nearly zero variance with Parameter 5 the next to least. These parameters could be discarded during the modellingdef get_VIF_Table(df,target):
from statsmodels.stats.outliers_influence import variance_inflation_factor
df = df.drop(target,axis=1).copy()
vif_data = pd.DataFrame()
vif_data["Feature"] = df.columns
vif_data["VIF"] =[variance_inflation_factor(df.values, i) for i in range(len(df.columns))]
return vif_data.sort_values(by='VIF',ascending=False)
get_VIF_Table(df,target)
| Feature | VIF | |
|---|---|---|
| 7 | Parameter 8 | 1479.287209 |
| 8 | Parameter 9 | 1070.967685 |
| 10 | Parameter 11 | 124.394866 |
| 0 | Parameter 1 | 74.452265 |
| 9 | Parameter 10 | 21.590621 |
| 1 | Parameter 2 | 17.060026 |
| 2 | Parameter 3 | 9.183495 |
| 4 | Parameter 5 | 6.554877 |
| 6 | Parameter 7 | 6.519699 |
| 5 | Parameter 6 | 6.442682 |
| 3 | Parameter 4 | 4.662992 |
Parameter 8, 9, 11,1,10 and 2 have very high VIFs i.e multi-collienarity.class DNN_Model() :
def __init__(self,type='regression'):
self.type = type
def create_model(self,in_shape,outs,**kwargs):
self.in_shape = in_shape
if kwargs != {}:
self.num_layers = kwargs['num_layers'] if 'num_layers' in kwargs else 1
self.batch_size = kwargs['batch_size'] if 'batch_size' in kwargs else 10
self.activation = kwargs['activation'] if 'activation' in kwargs else 'relu'
self.kernel_init = kwargs['kernel_init'] if 'kernel_init' in kwargs else 'uniform'
# self.optimizer = kwargs['opts'] if 'opts' in kwargs else tf.keras.optimizers.Adam(learning_rate=0.001)
self.neurons_per_layer = kwargs['neurons_per_layer'] if 'neurons_per_layer' in kwargs else [64]*self.num_layers
self.drop_rate_per_layer = kwargs['drop_rate_per_layer'] if 'drop_rate_per_layer' in kwargs else [0.2]*self.num_layers
self.n_epochs = kwargs['n_epochs'] if 'n_epochs' in kwargs else 50
self.arch = kwargs['arch'] if 'arch' in kwargs else 'B A D'
self.momentum = kwargs['momentum'] if 'meomentum' in kwargs else 0.99
self.epsilon = kwargs['epsilon'] if 'epsilon' in kwargs else 0.005
self.beta_init_std = kwargs['beta_init_std'] if 'beta_init_std' in kwargs else 0.05
self.gamma_init = kwargs['gamma_init'] if 'gamma_init' in kwargs else 0.9
self.center = kwargs['center'] if 'center' in kwargs else True
self.scale = kwargs['scale'] if 'scale' in kwargs else False
if 'opts' in kwargs:
# check if the kwargs is a class object or a dictionary
if hasattr(kwargs['opts'],'_name'):
self.optimizer = kwargs['opts']
else:
opt_conf = kwargs['opts']
use_opt = opt_conf['optimizer']
opt_kwargs = {k:opt_conf[k] for k in opt_conf.keys() if k != 'optimizer'}
try:
self.optimizer = getattr(tf.optimizers, opt_conf['optimizer'])(**opt_kwargs)
except:
self.optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
else:
self.optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
else:
self.num_layers = 1
self.batch_size = 10
self.activation = 'relu'
self.kernel_init = 'normal'
self.optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
self.neurons_per_layer = [32] * self.num_layers
self.drop_rate_per_layer = [0.2] * self.num_layers
self.n_epochs = 50
self.arch = 'B A D'
self.momentum = 0.99
self.epsilon = 0.005
self.beta_init_std = 0.05
self.gamma_init = 0.9
self.center = True
self.scale = False
if self.type == 'regression':
self.op_activation = 'linear'
self.outs = 1
elif self.type == 'classification':
self.op_activation = 'softmax'
self.outs = outs
inputs = Input(shape=self.in_shape)
for layer in range(self.num_layers):
if layer == 0 :
m = Dense(self.neurons_per_layer[layer],kernel_initializer = self.kernel_init)(inputs)
else:
m = Dense(self.neurons_per_layer[layer],kernel_initializer = self.kernel_init)(m)
for keys in self.arch.split(' '):
if keys == 'A':
m = Activation(self.activation)(m)
elif keys == 'B':
m = BatchNormalization(momentum = self.momentum,
epsilon = self.epsilon,
beta_initializer = RandomNormal(mean=0.0, stddev=self.beta_init_std),
gamma_initializer = Constant(value=self.gamma_init),
center = self.center,
scale = self.scale)(m)
elif keys == 'D':
m = Dropout(self.drop_rate_per_layer[layer])(m)
outputs = Dense(self.outs, activation=self.op_activation,kernel_initializer = self.kernel_init)(m)
model = Model(inputs=inputs, outputs=outputs)
return(model)
def model_summary(self,model):
return model.summary()
def train_model(self,model,X_train,y_train,loss,metric,callbacks=None,validation_split=0.2,validation_data=None,verbose=1):
self.model = model
self.loss = loss
self.metric = metric
self.model.compile(loss=self.loss,
optimizer=self.optimizer,
metrics=self.metric)
if callbacks == None:
callbacks = []
else :
callbacks = callbacks
if validation_data == None:
self.history = self.model.fit(X_train,
y_train,
epochs=self.n_epochs,
validation_split=validation_split,
callbacks=callbacks,
verbose=0)
else:
self.history = self.model.fit(X_train,
y_train,
epochs=self.n_epochs,
validation_data=validation_data,
callbacks=callbacks,
verbose=0)
return
def test_model(self,X_test,y_test,callbacks=None):
self.test_results = self.model.evaluate(X_test,y_test,batch_size=self.batch_size,callbacks=callbacks,verbose=0)
return self.test_results
class LivePlot(tf.keras.callbacks.Callback):
def __init__(self,refresh_rate=5,train_loss=None,train_metric=None):
self.validation_prefix = "val_"
self.refresh_rate = refresh_rate
self.train_loss = train_loss
self.val_loss = self.validation_prefix + train_loss
self.train_metric = train_metric
self.val_metric = self.validation_prefix + train_metric
# This function is called when the training begins
def on_train_begin(self, logs={}):
# Initialize the lists for holding the logs, losses and metrics
self.train_losses = []
self.train_metrics = []
self.val_losses = []
self.val_metrics = []
self.logs = []
# This function is called at the end of each epoch
def on_epoch_end(self, epoch, logs={}):
"""
Calculates and plots loss and metrics
"""
# Extract from the log
log_train_loss = logs.get(self.train_loss)
log_train_metric = logs.get(self.train_metric)
log_val_loss = logs.get(self.val_loss)
log_val_metric = logs.get(self.val_metric)
# Append the logs, losses and accuracies to the lists
self.logs.append(logs)
self.train_losses.append(log_train_loss)
self.train_metrics.append(log_train_metric)
self.val_losses.append(log_val_loss)
self.val_metrics.append(log_val_metric)
# Plots every nth epoch
if epoch > 0 and epoch%self.refresh_rate == 0:
fig, ax = plt.subplots(1,2,figsize=(20,6))
clear_output(wait=True)
N = np.arange(0, len(self.train_losses))
sns.lineplot(x=N,y=self.train_losses,ax=ax[0],legend='brief',label=self.train_loss)
sns.lineplot(x=N, y = self.val_losses,ax=ax[0],legend='brief',label=self.val_loss)
ax[0].set_title(f'Loss over Epoch {epoch} - {log_train_loss:.3}/{log_val_loss:.3} - (T/V)\n',{'fontsize':20})
ax[0].set_xlabel('Epochs')
ax[0].set_ylabel(self.train_loss)
sns.lineplot(x=N,y=self.train_metrics,ax=ax[1],legend='brief',label=self.train_metric)
sns.lineplot(x=N,y=self.val_metrics,ax=ax[1],legend='brief',label=self.val_metric)
ax[1].set_title(f'Performance over Epoch {epoch} - {log_train_metric:.3}/{log_val_metric:.3} - (T/V) \n',{'fontsize':20})
ax[1].set_xlabel('Epochs')
ax[1].set_ylabel(self.train_metric)
plt.show()
baseline model/data will then be subjected to different levels of hyperparameter optimization.baseline the model, we will assume some basic hyperparameters as show below.Loss and Performance metric for regression problem will Mean Absolute Error (MAE)columns_to_drop = [target]
X = df.drop(columns_to_drop,axis=1)
y = df[target]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
X_train.shape, X_test.shape
((1279, 11), (320, 11))
## Baseline Params
kwargs = {'num_layers': 1,
'arch' :'B A D',
# 'neurons_per_layer':[64],
# 'batch_size': 10,
'drop_rate_per_layer': [0.],
# 'activation': 'relu',
'n_epochs': 50,
# 'kernel_init': 'uniform',
# 'lr': 0.001,
# 'opts': {'optimizer':'Adam'}
}
loss = 'mean_absolute_error'
metric = 'mean_absolute_error'
score_board = pd.DataFrame(columns=['Baseline','Loss','Metric','Notes'])
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1,**kwargs)
display(Markdown("### Base Model Summary"))
m_1.summary()
Model: "model_22" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_23 (InputLayer) [(None, 11)] 0 _________________________________________________________________ dense_86 (Dense) (None, 64) 768 _________________________________________________________________ batch_normalization_64 (Batc (None, 64) 192 _________________________________________________________________ activation_64 (Activation) (None, 64) 0 _________________________________________________________________ dropout_64 (Dropout) (None, 64) 0 _________________________________________________________________ dense_87 (Dense) (None, 1) 65 ================================================================= Total params: 1,025 Trainable params: 897 Non-trainable params: 128 _________________________________________________________________
dnn_1.__dict__
{'type': 'regression',
'in_shape': 11,
'num_layers': 1,
'batch_size': 10,
'activation': 'relu',
'kernel_init': 'uniform',
'neurons_per_layer': [64],
'drop_rate_per_layer': [0.0],
'n_epochs': 50,
'arch': 'B A D',
'momentum': 0.99,
'epsilon': 0.005,
'beta_init_std': 0.05,
'gamma_init': 0.9,
'center': True,
'scale': False,
'optimizer': <tensorflow.python.keras.optimizer_v2.gradient_descent.SGD at 0x7f30862485b0>,
'op_activation': 'linear',
'outs': 1}
batchNormalization, Activation and Dropout layers in that order.dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1,**kwargs)
callbacks = LivePlot(2,train_loss='loss',train_metric=metric)
# callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
display(Markdown(f"### Test Loss : {result[0]} Test Metric : {result[1]}"))
score_board = score_board.append({'Baseline':'No feature engineering/selection',
'Loss':result[0],
'Metric':result[1],
'Notes':f'Loss:{loss} - Metric: {metric}'},ignore_index=True)
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.647783 | 0.647783 | Loss:mean_absolute_error - Metric: mean_absolute_error |
We will change the original variables into quadratic variables `(degree 2)` and drop `Parameter 8` as it's very close to `1` and increasing the power will not create any significant variance.
dff = df.copy()
dff['Parameter 1'] = (dff['Parameter 1'])**2
dff['Parameter 2'] = (dff['Parameter 2'])**2
dff['Parameter 3'] = (dff['Parameter 3'])**2
dff['Parameter 4'] = (dff['Parameter 4'])**2
dff['Parameter 5'] = (dff['Parameter 5'])**2
dff['Parameter 6'] = (dff['Parameter 6'])**2
dff['Parameter 7'] = (dff['Parameter 7'])**2
dff['Parameter 8'] = (dff['Parameter 8'])**2
dff['Parameter 9'] = (dff['Parameter 9'])**2
dff['Parameter 10'] = (dff['Parameter 10'])**2
dff['Parameter 11'] = (dff['Parameter 11'])**2
dff.drop(['Parameter 8'],axis=1,inplace=True)
dff.head()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 9 | Parameter 10 | Parameter 11 | Signal_Strength | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 54.76 | 0.4900 | 0.0000 | 3.61 | 0.005776 | 121.0 | 1156.0 | 12.3201 | 0.3136 | 88.36 | 5 |
| 1 | 60.84 | 0.7744 | 0.0000 | 6.76 | 0.009604 | 625.0 | 4489.0 | 10.2400 | 0.4624 | 96.04 | 5 |
| 2 | 60.84 | 0.5776 | 0.0016 | 5.29 | 0.008464 | 225.0 | 2916.0 | 10.6276 | 0.4225 | 96.04 | 5 |
| 3 | 125.44 | 0.0784 | 0.3136 | 3.61 | 0.005625 | 289.0 | 3600.0 | 9.9856 | 0.3364 | 96.04 | 6 |
| 4 | 54.76 | 0.4900 | 0.0000 | 3.61 | 0.005776 | 121.0 | 1156.0 | 12.3201 | 0.3136 | 88.36 | 5 |
get_VIF_Table(dff,target)
| Feature | VIF | |
|---|---|---|
| 7 | Parameter 9 | 35.338024 |
| 9 | Parameter 11 | 28.248033 |
| 0 | Parameter 1 | 10.064286 |
| 2 | Parameter 3 | 4.784843 |
| 8 | Parameter 10 | 4.448403 |
| 1 | Parameter 2 | 4.084400 |
| 5 | Parameter 6 | 2.160246 |
| 6 | Parameter 7 | 1.921041 |
| 4 | Parameter 5 | 1.771888 |
| 3 | Parameter 4 | 1.381402 |
columns_to_drop = [target]
Xf = dff.drop(columns_to_drop,axis=1)
yf = dff[target]
Xf_train, Xf_test, yf_train, yf_test = train_test_split(Xf, yf, test_size=0.2, random_state=0)
Xf_train.shape, Xf_test.shape
((1279, 10), (320, 10))
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(Xf_train.shape[1],1,**kwargs)
callbacks = LivePlot(2,train_loss=loss,train_metric=metric)
# callbacks=[]
dnn_1.train_model(m_1,Xf_train,yf_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(Xf_test,yf_test)
display(Markdown(f"### Test Loss : {result[0]} Test Metric : {result[1]}"))
score_board = score_board.append({'Baseline':'Feature engineering only',
'Loss':result[0],
'Metric':result[1],
'Notes':f'Loss:{loss} - Metric: {metric}'},ignore_index=True)
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.647783 | 0.647783 | Loss:mean_absolute_error - Metric: mean_absolute_error |
| 1 | Feature engineering only | 0.582681 | 0.582681 | Loss:mean_absolute_error - Metric: mean_absolute_error |
from sklearn.ensemble import ExtraTreesRegressor
from sklearn.feature_selection import SelectFromModel
dfe = df.copy()
x_columns = dfe.drop([target],axis=1).columns
Xe, ye = dfe[x_columns], dfe[target]
print(f"Shape of original data : {Xe.shape}")
clf = ExtraTreesRegressor(n_estimators=50)
clf = clf.fit(Xe, ye)
model = SelectFromModel(clf, prefit=True)
Xe= model.transform(Xe)
print(f"Shape of transformed data after feature selection : {Xe.shape}")
feat_df = pd.DataFrame({'Features':x_columns, 'Importance':clf.feature_importances_}).sort_values(by='Importance',ascending=False)
feat_df
Shape of original data : (1599, 11) Shape of transformed data after feature selection : (1599, 3)
| Features | Importance | |
|---|---|---|
| 10 | Parameter 11 | 0.258305 |
| 1 | Parameter 2 | 0.137069 |
| 9 | Parameter 10 | 0.117569 |
| 6 | Parameter 7 | 0.074177 |
| 3 | Parameter 4 | 0.065771 |
| 2 | Parameter 3 | 0.062183 |
| 4 | Parameter 5 | 0.059076 |
| 8 | Parameter 9 | 0.058134 |
| 0 | Parameter 1 | 0.056376 |
| 7 | Parameter 8 | 0.056138 |
| 5 | Parameter 6 | 0.055201 |
features_chosen = feat_df['Features'][:Xe.shape[1]].values
features_chosen
array(['Parameter 11', 'Parameter 2', 'Parameter 10'], dtype=object)
Parameter 11, 2 and 10. This has siginificantly reduced the feature space.Xe = dfe[features_chosen]
ye = dfe[target]
Xe_train, Xe_test, ye_train, ye_test = train_test_split(Xe, ye, test_size=0.2, random_state=0)
Xe_train.shape, Xe_test.shape
((1279, 3), (320, 3))
scaled = StandardScaler().fit(Xe_train)
Xe_train = scaled.transform(Xe_train)
Xe_test = scaled.transform(Xe_test)
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(Xe_train.shape[1],1,**kwargs)
callbacks = LivePlot(2,train_loss=loss,train_metric=metric)
# callbacks=[]
dnn_1.train_model(m_1,Xe_train,ye_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(Xe_test,ye_test)
display(Markdown(f"### Test Loss : {result[0]} Test Metric : {result[1]}"))
score_board = score_board.append({'Baseline':'Feature selection only',
'Loss':result[0],
'Metric':result[1],
'Notes':f'Loss:{loss} - Metric: {metric}'},ignore_index=True)
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.647783 | 0.647783 | Loss:mean_absolute_error - Metric: mean_absolute_error |
| 1 | Feature engineering only | 0.582681 | 0.582681 | Loss:mean_absolute_error - Metric: mean_absolute_error |
| 2 | Feature selection only | 0.506802 | 0.506802 | Loss:mean_absolute_error - Metric: mean_absolute_error |
display(Markdown("#### Score Board from model baselining"))
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.647783 | 0.647783 | Loss:mean_absolute_error - Metric: mean_absolute_error |
| 1 | Feature engineering only | 0.582681 | 0.582681 | Loss:mean_absolute_error - Metric: mean_absolute_error |
| 2 | Feature selection only | 0.506802 | 0.506802 | Loss:mean_absolute_error - Metric: mean_absolute_error |
Based on the score board above, Loss/Metric is the least for the data with feature selection and hence we will use this as a baseline for our hyper parameter tuning and model training
Activation, BatchNormalization and Dropout.At each stage the parameter determined in the previous stage flows into the next one to override defaults
X_train,X_test,y_train,y_test = Xe_train, Xe_test, ye_train, ye_test
X_train.shape, X_test.shape
((1279, 3), (320, 3))
hp_score_board = pd.DataFrame(columns=['Phase','Best Value','Hyperparameters'])
def objective_arch(trial):
# A = Activation
# B = BatchNormalization
# D = Dropout
#['A B D'] = ['Activation','BarchNormalization','Dropout']
global loss,metric,X_train,X_test,y_train,y_test
epochs = 50
lr = 0.001
num_layers = trial.suggest_categorical('num_layers',[1,2,3,4,5,6,7])
arch = trial.suggest_categorical('arch',[' '.join(i) for i in permutations(['A','B','D'],3)])
neurons_per_layer = [trial.suggest_categorical(f'neuron_l{layers}',[10,32,64,128,256,1000]) for layers in range(1,num_layers+1)]
opts = 'adam'
if opts == 'sgd':
optimizer = tf.keras.optimizers.SGD(learning_rate=lr)
elif opts == 'adadelta':
optimizer = tf.keras.optimizers.Adadelta(learning_rate=lr)
elif opts == 'adam':
optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
elif opts == 'rmsprop':
optimizer = tf.keras.optimizers.RMSprop(learning_rate=lr)
kwargs = {'num_layers': num_layers,
'arch' :arch,
'neurons_per_layer':neurons_per_layer,
'batch_size': 10,
'drop_rate_per_layer': [0.2]*num_layers,
'activation': 'relu',
'n_epochs': epochs,
'kernel_init': 'uniform',
'lr': lr,
'opts': optimizer}
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1,**kwargs)
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(Xe_test,ye_test)
return result[0]
print(X_train.shape,X_test.shape,y_train.shape,y_test.shape)
study_arch = optuna.create_study(sampler=optuna.samplers.TPESampler())
study_arch.optimize(objective_arch,n_trials=800,n_jobs=-13)
[I 2021-03-10 16:01:28,936] A new study created in memory with name: no-name-0d4846cc-9c9e-4523-9d13-5a29273a5a5e [I 2021-03-10 16:01:42,609] Trial 0 finished with value: 0.7737525701522827 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256}. Best is trial 0 with value: 0.7737525701522827. [I 2021-03-10 16:01:44,546] Trial 3 finished with value: 0.48900213837623596 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 32}. Best is trial 3 with value: 0.48900213837623596. [I 2021-03-10 16:01:44,604] Trial 2 finished with value: 0.4480738639831543 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 128}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:01:49,119] Trial 1 finished with value: 0.46636801958084106 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:01:58,967] Trial 7 finished with value: 0.4740350842475891 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 10, 'neuron_l2': 64}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:01:59,423] Trial 6 finished with value: 0.7035084366798401 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:02,175] Trial 5 finished with value: 0.48530545830726624 and parameters: {'num_layers': 5, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 10}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:03,336] Trial 4 finished with value: 0.4779215455055237 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 64}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:17,375] Trial 11 finished with value: 0.49881014227867126 and parameters: {'num_layers': 4, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 32}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:20,556] Trial 10 finished with value: 0.6540077328681946 and parameters: {'num_layers': 5, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 128}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:20,610] Trial 8 finished with value: 0.7838914394378662 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 256, 'neuron_l5': 1000}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:23,182] Trial 9 finished with value: 0.54353928565979 and parameters: {'num_layers': 5, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 128, 'neuron_l5': 1000}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:31,362] Trial 14 finished with value: 0.5390098690986633 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:39,986] Trial 12 finished with value: 0.47763919830322266 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 256, 'neuron_l7': 32}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:43,641] Trial 13 finished with value: 0.45907893776893616 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 128}. Best is trial 2 with value: 0.4480738639831543. [I 2021-03-10 16:02:45,076] Trial 15 finished with value: 0.41305145621299744 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:02:47,711] Trial 16 finished with value: 0.5181885957717896 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:02:57,575] Trial 17 finished with value: 0.5142539143562317 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:04,304] Trial 19 finished with value: 0.5877478122711182 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:06,261] Trial 18 finished with value: 0.48206979036331177 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 64, 'neuron_l6': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:09,652] Trial 20 finished with value: 0.5792200565338135 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:15,128] Trial 21 finished with value: 0.4807712435722351 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:23,913] Trial 24 finished with value: 0.5816878080368042 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:26,023] Trial 23 finished with value: 0.5092644095420837 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:26,703] Trial 25 finished with value: 0.4611537456512451 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:29,926] Trial 22 finished with value: 0.4746345579624176 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 64, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:37,733] Trial 27 finished with value: 0.49916911125183105 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:39,124] Trial 28 finished with value: 0.48296791315078735 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:40,913] Trial 29 finished with value: 0.5744156837463379 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:03:50,036] Trial 26 finished with value: 0.5002373456954956 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:04,350] Trial 33 finished with value: 0.49402204155921936 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:05,136] Trial 30 finished with value: 0.4799136221408844 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:09,002] Trial 31 finished with value: 0.5456055402755737 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:10,167] Trial 32 finished with value: 0.47628870606422424 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:16,982] Trial 34 finished with value: 0.4923034608364105 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:17,871] Trial 35 finished with value: 0.4455246925354004 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:22,479] Trial 36 finished with value: 0.5587488412857056 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:24,202] Trial 37 finished with value: 0.4867404103279114 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:41,260] Trial 39 finished with value: 0.4583727717399597 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:42,178] Trial 38 finished with value: 0.4712608754634857 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:44,113] Trial 41 finished with value: 0.7452849745750427 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:04:44,161] Trial 40 finished with value: 0.46820878982543945 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:03,323] Trial 42 finished with value: 0.5062218904495239 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:05,006] Trial 44 finished with value: 0.45250964164733887 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:05,108] Trial 45 finished with value: 0.43916377425193787 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:06,409] Trial 43 finished with value: 0.6234015226364136 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:22,508] Trial 46 finished with value: 0.46001696586608887 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:25,355] Trial 48 finished with value: 0.4419183135032654 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:25,360] Trial 47 finished with value: 0.44580382108688354 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:26,441] Trial 49 finished with value: 0.44356870651245117 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:34,085] Trial 50 finished with value: 0.4506594240665436 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:47,192] Trial 52 finished with value: 0.4327104985713959 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:47,671] Trial 53 finished with value: 0.4340642988681793 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:47,733] Trial 51 finished with value: 0.4237282872200012 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:05:51,363] Trial 54 finished with value: 0.42048779129981995 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:09,251] Trial 57 finished with value: 0.45669612288475037 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:09,323] Trial 56 finished with value: 0.43558263778686523 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:09,336] Trial 55 finished with value: 0.4437227249145508 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:09,337] Trial 58 finished with value: 0.4280930459499359 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:27,664] Trial 62 finished with value: 1.0769413709640503 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:28,049] Trial 61 finished with value: 0.48128312826156616 and parameters: {'num_layers': 5, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:29,783] Trial 60 finished with value: 0.5924240350723267 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:30,322] Trial 59 finished with value: 0.4719841480255127 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:48,508] Trial 63 finished with value: 0.47580140829086304 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:50,961] Trial 64 finished with value: 0.43676406145095825 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:51,067] Trial 65 finished with value: 0.4517636299133301 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:06:51,405] Trial 66 finished with value: 0.49177664518356323 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:09,090] Trial 67 finished with value: 0.45910367369651794 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:11,029] Trial 70 finished with value: 0.49602586030960083 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 128, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:11,083] Trial 69 finished with value: 0.5096713900566101 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 128, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:11,116] Trial 68 finished with value: 0.42755126953125 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:26,208] Trial 73 finished with value: 0.4951001703739166 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:29,869] Trial 74 finished with value: 0.4877029359340668 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:31,288] Trial 71 finished with value: 0.4928012490272522 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 128, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:32,103] Trial 72 finished with value: 0.4974219799041748 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:43,971] Trial 75 finished with value: 0.42768803238868713 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:50,071] Trial 77 finished with value: 0.5067685842514038 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:50,159] Trial 78 finished with value: 0.4797491431236267 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:07:51,240] Trial 76 finished with value: 0.4756235182285309 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:00,195] Trial 81 finished with value: 0.46485716104507446 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:00,200] Trial 80 finished with value: 0.4610517919063568 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:00,712] Trial 82 finished with value: 0.45610517263412476 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:00,754] Trial 79 finished with value: 0.4675024151802063 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:19,745] Trial 83 finished with value: 0.6356145143508911 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:19,781] Trial 86 finished with value: 0.4552457928657532 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:19,793] Trial 84 finished with value: 0.5557571649551392 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:19,794] Trial 85 finished with value: 0.5180668830871582 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:35,638] Trial 87 finished with value: 0.44162505865097046 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:42,894] Trial 89 finished with value: 0.47443240880966187 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:50,292] Trial 90 finished with value: 0.4516991972923279 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:08:50,333] Trial 88 finished with value: 0.48615044355392456 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:01,853] Trial 91 finished with value: 2.2949743270874023 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:01,945] Trial 92 finished with value: 1.6556336879730225 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:10,357] Trial 94 finished with value: 0.4778634011745453 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:13,741] Trial 93 finished with value: 1.1601892709732056 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:21,615] Trial 96 finished with value: 0.45451799035072327 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:21,620] Trial 95 finished with value: 0.4785881042480469 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:28,825] Trial 97 finished with value: 0.5641476511955261 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:34,011] Trial 98 finished with value: 0.5269361734390259 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 64, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:37,423] Trial 99 finished with value: 0.4373491406440735 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:37,600] Trial 100 finished with value: 0.4528927803039551 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:43,043] Trial 101 finished with value: 0.5262755155563354 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:49,457] Trial 102 finished with value: 0.44778886437416077 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:51,226] Trial 103 finished with value: 0.445981502532959 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:53,933] Trial 104 finished with value: 0.4722033441066742 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:09:57,216] Trial 105 finished with value: 0.502338707447052 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:06,469] Trial 106 finished with value: 0.4492056369781494 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:09,368] Trial 107 finished with value: 0.4502280652523041 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:14,083] Trial 108 finished with value: 0.44055262207984924 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:16,879] Trial 109 finished with value: 0.4320172667503357 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:27,196] Trial 110 finished with value: 0.46641987562179565 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:31,859] Trial 111 finished with value: 0.49326252937316895 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:35,428] Trial 112 finished with value: 0.45845264196395874 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:36,962] Trial 113 finished with value: 0.4606474041938782 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:45,571] Trial 114 finished with value: 0.4519152045249939 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:53,456] Trial 115 finished with value: 0.4139537811279297 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:58,128] Trial 117 finished with value: 0.474601686000824 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:10:58,946] Trial 116 finished with value: 0.4532657265663147 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:06,736] Trial 118 finished with value: 0.5097494721412659 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:17,520] Trial 119 finished with value: 0.49799662828445435 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:22,407] Trial 121 finished with value: 0.505437970161438 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:25,942] Trial 120 finished with value: 0.4195282459259033 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:27,184] Trial 122 finished with value: 0.5949713587760925 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:40,075] Trial 123 finished with value: 0.41677436232566833 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:44,688] Trial 124 finished with value: 0.42561230063438416 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:54,845] Trial 126 finished with value: 0.493478000164032 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:11:55,537] Trial 125 finished with value: 0.4266439974308014 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:07,704] Trial 127 finished with value: 0.5071808099746704 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:11,215] Trial 128 finished with value: 0.4982142448425293 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:17,931] Trial 129 finished with value: 0.4517790675163269 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:18,245] Trial 130 finished with value: 0.4535411298274994 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:22,389] Trial 132 finished with value: 0.4744475483894348 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:29,477] Trial 131 finished with value: 0.43861550092697144 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:35,401] Trial 134 finished with value: 0.6053562760353088 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:43,566] Trial 133 finished with value: 0.4833126664161682 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 256, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:46,963] Trial 135 finished with value: 0.4316674768924713 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:54,989] Trial 136 finished with value: 0.4902823567390442 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 256, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:12:59,919] Trial 137 finished with value: 0.4538050591945648 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 256, 'neuron_l7': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:04,038] Trial 138 finished with value: 0.45936551690101624 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:10,636] Trial 139 finished with value: 0.5104591250419617 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:18,857] Trial 140 finished with value: 0.5003809332847595 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:24,216] Trial 141 finished with value: 0.44644150137901306 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:30,093] Trial 142 finished with value: 0.45939356088638306 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:34,806] Trial 144 finished with value: 0.4137859344482422 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:36,867] Trial 143 finished with value: 0.46961697936058044 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:41,285] Trial 145 finished with value: 0.4366195797920227 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:47,115] Trial 146 finished with value: 0.48985451459884644 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:52,194] Trial 149 finished with value: 0.4392739236354828 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:54,574] Trial 147 finished with value: 0.46043339371681213 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:57,093] Trial 148 finished with value: 0.45249098539352417 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:13:58,581] Trial 150 finished with value: 0.42558327317237854 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:10,054] Trial 154 finished with value: 0.4409104883670807 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:12,009] Trial 151 finished with value: 0.47159019112586975 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:12,960] Trial 153 finished with value: 0.46788129210472107 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:13,444] Trial 152 finished with value: 0.49153995513916016 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:22,162] Trial 155 finished with value: 0.5157551169395447 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:24,830] Trial 156 finished with value: 0.439117431640625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:24,992] Trial 157 finished with value: 0.4521830976009369 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:25,639] Trial 158 finished with value: 0.44482988119125366 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:34,434] Trial 159 finished with value: 0.4685630202293396 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:46,147] Trial 160 finished with value: 0.430389404296875 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:46,179] Trial 161 finished with value: 0.4442787766456604 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:46,183] Trial 162 finished with value: 0.4321044087409973 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:14:51,309] Trial 163 finished with value: 0.4525819420814514 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:06,890] Trial 166 finished with value: 0.46131110191345215 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:06,935] Trial 165 finished with value: 0.44381481409072876 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:06,951] Trial 164 finished with value: 0.4455680251121521 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:08,946] Trial 167 finished with value: 0.4459327757358551 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:27,898] Trial 168 finished with value: 0.43945103883743286 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:28,938] Trial 170 finished with value: 0.490664541721344 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:30,827] Trial 169 finished with value: 0.4513053297996521 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:30,831] Trial 171 finished with value: 0.5269516706466675 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:37,997] Trial 173 finished with value: 0.4775429368019104 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:38,217] Trial 172 finished with value: 0.44481998682022095 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:51,753] Trial 175 finished with value: 0.41734370589256287 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:52,829] Trial 174 finished with value: 0.43571218848228455 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:56,356] Trial 176 finished with value: 0.46705490350723267 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:15:56,364] Trial 177 finished with value: 0.4693962037563324 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:12,285] Trial 178 finished with value: 0.4348269999027252 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:13,073] Trial 179 finished with value: 0.43109169602394104 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:15,383] Trial 180 finished with value: 1.1830756664276123 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:15,625] Trial 181 finished with value: 0.5013253092765808 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 64, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:34,370] Trial 183 finished with value: 0.41653355956077576 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:34,906] Trial 185 finished with value: 0.4748947024345398 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:34,972] Trial 184 finished with value: 0.48312515020370483 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 64, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:35,020] Trial 182 finished with value: 0.6295846700668335 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 64, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:54,949] Trial 187 finished with value: 0.45185714960098267 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:56,136] Trial 186 finished with value: 0.4311144948005676 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:56,175] Trial 189 finished with value: 0.44508466124534607 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:16:56,187] Trial 188 finished with value: 0.44562584161758423 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:16,321] Trial 190 finished with value: 0.43498358130455017 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:18,983] Trial 193 finished with value: 0.439785897731781 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:19,031] Trial 192 finished with value: 0.44338124990463257 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:19,061] Trial 191 finished with value: 0.42263323068618774 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:38,425] Trial 194 finished with value: 0.43654337525367737 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:41,460] Trial 195 finished with value: 0.4470607340335846 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:41,547] Trial 196 finished with value: 0.4459393620491028 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:41,638] Trial 197 finished with value: 0.4312805235385895 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:17:59,917] Trial 200 finished with value: 0.5856964588165283 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000}. Best is trial 15 with value: 0.41305145621299744. [I 2021-03-10 16:18:02,216] Trial 198 finished with value: 0.41268259286880493 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:04,160] Trial 199 finished with value: 0.44815462827682495 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:04,854] Trial 201 finished with value: 0.4533790051937103 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:20,199] Trial 202 finished with value: 0.4375961422920227 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:24,318] Trial 203 finished with value: 0.6014668941497803 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:27,145] Trial 204 finished with value: 0.44189175963401794 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:27,166] Trial 205 finished with value: 0.4165154993534088 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:40,276] Trial 206 finished with value: 0.41413307189941406 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:46,646] Trial 207 finished with value: 0.4260975420475006 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:50,425] Trial 209 finished with value: 0.4358143210411072 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:18:50,553] Trial 208 finished with value: 0.43774884939193726 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:03,027] Trial 212 finished with value: 0.47403138875961304 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 32}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:04,062] Trial 213 finished with value: 0.5083101391792297 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:05,464] Trial 210 finished with value: 0.43480801582336426 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:13,470] Trial 211 finished with value: 0.48867353796958923 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:26,821] Trial 216 finished with value: 0.4504578113555908 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:31,655] Trial 214 finished with value: 0.428072452545166 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:31,673] Trial 215 finished with value: 0.42198020219802856 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:33,584] Trial 217 finished with value: 0.5114089250564575 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:42,822] Trial 218 finished with value: 0.4656847417354584 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:54,691] Trial 221 finished with value: 0.4494965970516205 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:55,063] Trial 219 finished with value: 0.43082696199417114 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:19:59,271] Trial 220 finished with value: 0.45451679825782776 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:05,348] Trial 222 finished with value: 0.43652400374412537 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:19,062] Trial 223 finished with value: 0.4659012258052826 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:23,004] Trial 224 finished with value: 0.4466947615146637 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:25,348] Trial 225 finished with value: 0.41483479738235474 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:29,166] Trial 226 finished with value: 0.4277971386909485 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:41,146] Trial 227 finished with value: 0.42887893319129944 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:54,024] Trial 229 finished with value: 0.48777318000793457 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:54,466] Trial 228 finished with value: 0.5052119493484497 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:20:56,294] Trial 230 finished with value: 0.4456575810909271 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:21:03,865] Trial 231 finished with value: 0.4370911121368408 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:21:24,660] Trial 234 finished with value: 0.42723774909973145 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 198 with value: 0.41268259286880493. [I 2021-03-10 16:21:25,413] Trial 232 finished with value: 0.39887046813964844 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:21:25,475] Trial 233 finished with value: 0.440359503030777 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:21:29,205] Trial 235 finished with value: 0.4123941957950592 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:21:53,939] Trial 236 finished with value: 0.4650394022464752 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:21:55,740] Trial 238 finished with value: 0.43905526399612427 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:21:57,878] Trial 237 finished with value: 0.47236427664756775 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:21:58,071] Trial 239 finished with value: 0.5049388408660889 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:11,231] Trial 241 finished with value: 0.4509221613407135 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:13,812] Trial 243 finished with value: 0.6024575233459473 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:19,642] Trial 242 finished with value: 0.4525562822818756 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:19,722] Trial 240 finished with value: 0.4400365948677063 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:38,216] Trial 244 finished with value: 0.42295974493026733 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:40,642] Trial 245 finished with value: 0.4188782572746277 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:47,288] Trial 247 finished with value: 0.45592623949050903 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:22:47,313] Trial 246 finished with value: 0.4655572772026062 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:03,931] Trial 248 finished with value: 0.45960912108421326 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:06,142] Trial 249 finished with value: 0.5312504172325134 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:14,562] Trial 250 finished with value: 0.41850632429122925 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:14,563] Trial 251 finished with value: 0.4494083523750305 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:29,302] Trial 252 finished with value: 0.43599286675453186 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:30,822] Trial 253 finished with value: 0.41553306579589844 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:43,343] Trial 255 finished with value: 0.4344468116760254 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:43,358] Trial 254 finished with value: 0.4629786014556885 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:50,231] Trial 256 finished with value: 0.4609522223472595 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:23:55,366] Trial 257 finished with value: 0.47968992590904236 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:00,703] Trial 258 finished with value: 0.44349735975265503 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:05,491] Trial 259 finished with value: 0.47759246826171875 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:10,518] Trial 260 finished with value: 0.4576601982116699 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:15,341] Trial 261 finished with value: 0.422334760427475 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:22,460] Trial 262 finished with value: 0.5042392611503601 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:28,406] Trial 263 finished with value: 0.4949965476989746 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:30,023] Trial 264 finished with value: 0.4808464050292969 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:35,245] Trial 265 finished with value: 0.4210526943206787 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:38,489] Trial 267 finished with value: 0.4854200780391693 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:40,472] Trial 268 finished with value: 0.47668832540512085 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:43,960] Trial 266 finished with value: 0.42953962087631226 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:24:46,305] Trial 269 finished with value: 0.4788086414337158 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:02,309] Trial 270 finished with value: 0.44994640350341797 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:05,177] Trial 271 finished with value: 0.46643370389938354 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:11,944] Trial 272 finished with value: 0.5163089036941528 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:11,946] Trial 273 finished with value: 0.48984774947166443 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:29,221] Trial 274 finished with value: 0.42019811272621155 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:36,111] Trial 276 finished with value: 0.4345523416996002 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:37,011] Trial 275 finished with value: 0.4505729079246521 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:40,081] Trial 277 finished with value: 0.5133987665176392 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:51,945] Trial 280 finished with value: 0.46950164437294006 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:52,652] Trial 278 finished with value: 0.4382871985435486 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:25:55,554] Trial 281 finished with value: 1.4286651611328125 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:01,385] Trial 279 finished with value: 0.4569370746612549 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:17,593] Trial 283 finished with value: 1.2011319398880005 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:17,600] Trial 282 finished with value: 0.7200188636779785 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:21,196] Trial 284 finished with value: 0.43966445326805115 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:21,874] Trial 285 finished with value: 0.4377914369106293 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:42,749] Trial 289 finished with value: 0.45062685012817383 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:47,236] Trial 286 finished with value: 0.46837520599365234 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:47,241] Trial 287 finished with value: 0.47468671202659607 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:48,933] Trial 288 finished with value: 0.46365928649902344 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:58,402] Trial 292 finished with value: 0.4313027858734131 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:26:58,757] Trial 291 finished with value: 0.46586036682128906 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:00,178] Trial 290 finished with value: 0.41390109062194824 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:01,689] Trial 293 finished with value: 0.4596746861934662 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:18,985] Trial 296 finished with value: 0.41818803548812866 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:19,727] Trial 295 finished with value: 0.4445522725582123 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:19,824] Trial 297 finished with value: 0.4506891667842865 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:22,418] Trial 294 finished with value: 0.45957455039024353 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:38,236] Trial 298 finished with value: 0.4744701385498047 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:39,661] Trial 301 finished with value: 0.43066030740737915 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:39,703] Trial 299 finished with value: 0.4331943988800049 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:39,949] Trial 300 finished with value: 0.46884089708328247 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:57,152] Trial 302 finished with value: 0.4231489598751068 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:58,550] Trial 304 finished with value: 0.4698377549648285 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:58,555] Trial 303 finished with value: 0.44612306356430054 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:27:58,570] Trial 305 finished with value: 0.42698797583580017 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:17,115] Trial 306 finished with value: 0.4582946300506592 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:26,847] Trial 308 finished with value: 0.5055053234100342 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:28,207] Trial 307 finished with value: 0.6214226484298706 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:28,260] Trial 309 finished with value: 0.4219669699668884 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:40,107] Trial 310 finished with value: 0.4889191687107086 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:54,392] Trial 313 finished with value: 0.4779226779937744 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:57,115] Trial 311 finished with value: 0.45236238837242126 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:57,310] Trial 312 finished with value: 0.4999867081642151 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:28:59,611] Trial 314 finished with value: 0.4990712106227875 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:14,009] Trial 315 finished with value: 0.42229652404785156 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:16,731] Trial 316 finished with value: 0.5025405883789062 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:16,850] Trial 317 finished with value: 0.4511790871620178 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:18,374] Trial 318 finished with value: 0.4418370723724365 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:33,642] Trial 319 finished with value: 0.45444589853286743 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:35,165] Trial 322 finished with value: 0.6147706508636475 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:35,192] Trial 321 finished with value: 0.46906670928001404 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:36,776] Trial 320 finished with value: 0.5307341814041138 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:29:55,779] Trial 323 finished with value: 0.43141546845436096 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:00,497] Trial 324 finished with value: 0.449832022190094 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:06,161] Trial 325 finished with value: 0.4411841928958893 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:07,498] Trial 326 finished with value: 0.45405903458595276 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:23,053] Trial 328 finished with value: 0.43417221307754517 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:27,603] Trial 327 finished with value: 0.4562966227531433 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:30,252] Trial 329 finished with value: 0.422978937625885 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:30,504] Trial 330 finished with value: 0.4665113389492035 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:41,857] Trial 331 finished with value: 0.5135194659233093 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:43,343] Trial 332 finished with value: 0.5027081370353699 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:46,931] Trial 334 finished with value: 0.45035499334335327 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:30:50,344] Trial 333 finished with value: 0.4574580788612366 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:08,172] Trial 337 finished with value: 0.48255378007888794 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:10,005] Trial 335 finished with value: 0.47874054312705994 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:10,078] Trial 336 finished with value: 0.5066896080970764 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:13,857] Trial 338 finished with value: 0.4914659559726715 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:39,029] Trial 341 finished with value: 0.4517122209072113 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:40,515] Trial 339 finished with value: 0.44674235582351685 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:40,618] Trial 340 finished with value: 0.4244886338710785 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:31:41,759] Trial 342 finished with value: 0.44254836440086365 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:02,588] Trial 343 finished with value: 0.44275322556495667 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:03,220] Trial 345 finished with value: 0.48172974586486816 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:03,237] Trial 346 finished with value: 0.5210883617401123 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:04,248] Trial 344 finished with value: 0.41969960927963257 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:24,454] Trial 350 finished with value: 0.44200053811073303 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:26,254] Trial 348 finished with value: 0.43643832206726074 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:26,284] Trial 349 finished with value: 0.4382001459598541 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:26,353] Trial 347 finished with value: 0.4457198679447174 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:46,581] Trial 351 finished with value: 0.4190663695335388 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:53,292] Trial 353 finished with value: 0.5392740368843079 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:54,531] Trial 352 finished with value: 0.45248252153396606 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:32:54,537] Trial 354 finished with value: 0.5457292795181274 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:08,223] Trial 355 finished with value: 0.647365391254425 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:17,121] Trial 357 finished with value: 0.44559231400489807 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:17,122] Trial 358 finished with value: 0.40289315581321716 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:20,931] Trial 356 finished with value: 0.4232451915740967 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:25,597] Trial 359 finished with value: 0.4183433949947357 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:40,147] Trial 361 finished with value: 0.4527660310268402 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:40,391] Trial 360 finished with value: 0.4577513337135315 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:43,283] Trial 362 finished with value: 0.49202316999435425 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:33:45,595] Trial 363 finished with value: 0.4889853000640869 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:00,656] Trial 364 finished with value: 0.45089855790138245 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:01,509] Trial 365 finished with value: 0.4314803183078766 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:03,527] Trial 366 finished with value: 0.44299978017807007 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:04,657] Trial 367 finished with value: 0.43953847885131836 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:12,919] Trial 370 finished with value: 1.4570099115371704 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:24,277] Trial 369 finished with value: 0.43053174018859863 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:25,371] Trial 368 finished with value: 0.46386218070983887 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:28,935] Trial 371 finished with value: 0.46361613273620605 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:32,083] Trial 372 finished with value: 0.44981861114501953 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:54,346] Trial 373 finished with value: 0.4425402581691742 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:56,900] Trial 374 finished with value: 0.4219082295894623 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:59,146] Trial 376 finished with value: 0.474200576543808 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:34:59,439] Trial 375 finished with value: 0.46247944235801697 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:21,054] Trial 378 finished with value: 0.45921698212623596 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:21,707] Trial 377 finished with value: 0.4530976712703705 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:23,294] Trial 379 finished with value: 0.4353605806827545 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:23,712] Trial 380 finished with value: 0.46551817655563354 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:43,031] Trial 381 finished with value: 0.44972196221351624 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 128, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:44,420] Trial 383 finished with value: 0.46658000349998474 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:45,528] Trial 382 finished with value: 0.4472506642341614 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:35:46,638] Trial 384 finished with value: 0.5894293189048767 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:06,261] Trial 385 finished with value: 0.46271833777427673 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:07,043] Trial 386 finished with value: 0.44612860679626465 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:08,918] Trial 387 finished with value: 0.4327614903450012 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:08,966] Trial 388 finished with value: 0.4326809048652649 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:20,426] Trial 390 finished with value: 0.49604788422584534 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:21,776] Trial 391 finished with value: 0.506040632724762 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:29,374] Trial 389 finished with value: 0.4477027952671051 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:31,624] Trial 392 finished with value: 0.42527809739112854 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:42,630] Trial 393 finished with value: 0.4556608200073242 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:44,553] Trial 394 finished with value: 0.5793924927711487 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:50,425] Trial 396 finished with value: 0.4543643593788147 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:36:50,996] Trial 395 finished with value: 0.43360012769699097 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:00,653] Trial 398 finished with value: 0.4437169134616852 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:03,968] Trial 397 finished with value: 0.4519471526145935 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:07,776] Trial 399 finished with value: 1.155959963798523 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:11,825] Trial 400 finished with value: 1.4263114929199219 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:22,634] Trial 401 finished with value: 1.1527931690216064 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:27,084] Trial 402 finished with value: 0.43673309683799744 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:33,084] Trial 405 finished with value: 0.44359827041625977 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:33,150] Trial 403 finished with value: 0.4347042143344879 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:35,644] Trial 404 finished with value: 0.45409202575683594 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:38,385] Trial 406 finished with value: 0.45061221718788147 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:37:59,421] Trial 407 finished with value: 0.4236966669559479 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:01,370] Trial 409 finished with value: 0.4366585612297058 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:01,445] Trial 410 finished with value: 0.5439316630363464 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:01,500] Trial 408 finished with value: 0.4340621829032898 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:26,130] Trial 411 finished with value: 0.5069334506988525 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:28,282] Trial 414 finished with value: 0.4896213114261627 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:28,311] Trial 413 finished with value: 0.45545855164527893 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:30,729] Trial 412 finished with value: 0.4797746539115906 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:38:59,644] Trial 415 finished with value: 0.5100833177566528 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:01,958] Trial 416 finished with value: 0.4583900570869446 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:03,770] Trial 417 finished with value: 0.4383170008659363 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:04,580] Trial 418 finished with value: 0.43585267663002014 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:26,933] Trial 420 finished with value: 0.4030977189540863 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:28,164] Trial 422 finished with value: 0.4533619284629822 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:28,341] Trial 421 finished with value: 0.4390995502471924 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:28,413] Trial 419 finished with value: 0.4410758912563324 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:50,345] Trial 423 finished with value: 0.4507018029689789 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:51,479] Trial 424 finished with value: 0.41635990142822266 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:51,507] Trial 425 finished with value: 0.47270941734313965 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:39:51,559] Trial 426 finished with value: 0.4314325749874115 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:12,736] Trial 427 finished with value: 0.4309559762477875 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:13,766] Trial 429 finished with value: 0.42448925971984863 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:14,936] Trial 428 finished with value: 0.44052982330322266 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:14,974] Trial 430 finished with value: 0.44029808044433594 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:36,798] Trial 432 finished with value: 0.4204885959625244 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:37,275] Trial 433 finished with value: 0.44323739409446716 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:37,969] Trial 434 finished with value: 0.4470018744468689 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:40:38,373] Trial 431 finished with value: 0.477177232503891 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:00,001] Trial 435 finished with value: 0.42072898149490356 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:04,542] Trial 438 finished with value: 0.4567844271659851 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:07,672] Trial 437 finished with value: 0.4653478264808655 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:11,901] Trial 436 finished with value: 0.45991963148117065 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:23,701] Trial 439 finished with value: 0.4728643298149109 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:29,607] Trial 441 finished with value: 0.4353572726249695 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:30,866] Trial 440 finished with value: 0.44537824392318726 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:33,499] Trial 442 finished with value: 0.44588667154312134 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:43,811] Trial 443 finished with value: 0.4472213685512543 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:57,132] Trial 445 finished with value: 0.4589046537876129 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:57,165] Trial 444 finished with value: 0.43282294273376465 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:41:58,863] Trial 446 finished with value: 0.43367379903793335 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:02,274] Trial 447 finished with value: 0.433797687292099 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:22,665] Trial 449 finished with value: 0.4367274343967438 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:24,941] Trial 448 finished with value: 0.4525013566017151 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:30,830] Trial 450 finished with value: 0.42510247230529785 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:33,510] Trial 451 finished with value: 0.4760758876800537 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:44,820] Trial 454 finished with value: 0.5039633512496948 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:46,777] Trial 455 finished with value: 0.45548027753829956 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:52,809] Trial 452 finished with value: 0.48531728982925415 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:42:55,827] Trial 453 finished with value: 0.4814504086971283 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:03,403] Trial 457 finished with value: 0.4459574818611145 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:09,225] Trial 458 finished with value: 0.4737720489501953 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:09,726] Trial 456 finished with value: 0.4303213953971863 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:12,025] Trial 459 finished with value: 0.49718984961509705 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:21,912] Trial 460 finished with value: 0.42916592955589294 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:26,555] Trial 461 finished with value: 0.4620351791381836 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:32,784] Trial 462 finished with value: 0.4297163486480713 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:32,874] Trial 463 finished with value: 0.4424210488796234 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:44,716] Trial 467 finished with value: 0.44758549332618713 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:51,129] Trial 464 finished with value: 0.47921308875083923 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:43:58,976] Trial 465 finished with value: 0.4775596261024475 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:03,581] Trial 466 finished with value: 0.46656355261802673 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:12,436] Trial 468 finished with value: 0.5086750388145447 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:19,192] Trial 469 finished with value: 0.43366512656211853 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:21,420] Trial 470 finished with value: 0.4651802182197571 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:27,264] Trial 471 finished with value: 0.4881201684474945 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 32, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:36,257] Trial 472 finished with value: 0.41193968057632446 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:47,452] Trial 473 finished with value: 0.4317382276058197 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:52,027] Trial 474 finished with value: 0.45237451791763306 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:44:55,213] Trial 475 finished with value: 0.45002803206443787 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:00,267] Trial 476 finished with value: 0.43815454840660095 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:15,829] Trial 477 finished with value: 0.466619074344635 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:23,166] Trial 478 finished with value: 0.41755765676498413 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:25,384] Trial 479 finished with value: 0.4284175932407379 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:26,930] Trial 480 finished with value: 0.5433085560798645 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:39,949] Trial 481 finished with value: 0.4332924485206604 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:51,334] Trial 482 finished with value: 0.49540311098098755 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:54,064] Trial 483 finished with value: 0.5062331557273865 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:45:55,097] Trial 484 finished with value: 0.4189114570617676 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:03,336] Trial 485 finished with value: 0.47167831659317017 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:19,405] Trial 486 finished with value: 0.4334842562675476 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:23,372] Trial 487 finished with value: 0.4332743287086487 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:23,384] Trial 488 finished with value: 0.4277594983577728 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:27,463] Trial 489 finished with value: 0.42432230710983276 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:43,573] Trial 492 finished with value: 0.4790704846382141 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:46,717] Trial 490 finished with value: 0.46340876817703247 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:50,950] Trial 491 finished with value: 0.4369412064552307 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:46:52,158] Trial 493 finished with value: 0.4552222192287445 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:05,055] Trial 497 finished with value: 0.4706001281738281 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:11,653] Trial 494 finished with value: 0.4474734365940094 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:15,056] Trial 495 finished with value: 0.46943527460098267 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:16,498] Trial 496 finished with value: 0.4376205503940582 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:29,388] Trial 498 finished with value: 0.4531609117984772 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:40,003] Trial 499 finished with value: 0.43471288681030273 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:43,239] Trial 501 finished with value: 0.6598436832427979 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 128, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:43,891] Trial 500 finished with value: 0.4846460223197937 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 128, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:47:48,275] Trial 502 finished with value: 0.4595244824886322 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:03,110] Trial 503 finished with value: 0.4459715485572815 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:07,652] Trial 504 finished with value: 0.7119919061660767 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:10,751] Trial 505 finished with value: 0.4582390785217285 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:14,244] Trial 506 finished with value: 2.1850247383117676 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:28,360] Trial 507 finished with value: 1.3279062509536743 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:33,316] Trial 508 finished with value: 0.45703354477882385 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:35,466] Trial 509 finished with value: 0.4299057424068451 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:36,939] Trial 510 finished with value: 0.4263380169868469 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:48,041] Trial 511 finished with value: 0.4242623746395111 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:56,203] Trial 513 finished with value: 0.4505079686641693 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:58,639] Trial 514 finished with value: 0.43142789602279663 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:48:58,950] Trial 512 finished with value: 0.4235835671424866 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:06,136] Trial 515 finished with value: 0.4769613742828369 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:19,793] Trial 516 finished with value: 0.41736021637916565 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:21,939] Trial 517 finished with value: 0.4654909670352936 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:22,906] Trial 518 finished with value: 0.46273255348205566 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:26,041] Trial 519 finished with value: 0.4450189173221588 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:32,442] Trial 520 finished with value: 0.4652668535709381 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:36,660] Trial 521 finished with value: 0.4737643301486969 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:37,165] Trial 522 finished with value: 0.46074342727661133 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:49:39,702] Trial 523 finished with value: 0.4293603301048279 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:00,324] Trial 524 finished with value: 0.41945576667785645 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:03,379] Trial 525 finished with value: 0.47950369119644165 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:03,450] Trial 527 finished with value: 0.42854243516921997 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:03,525] Trial 526 finished with value: 0.4424399435520172 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:20,548] Trial 529 finished with value: 0.42576831579208374 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:22,302] Trial 530 finished with value: 0.4640423357486725 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:26,134] Trial 531 finished with value: 0.4444923400878906 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:27,863] Trial 528 finished with value: 0.46558666229248047 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:43,155] Trial 532 finished with value: 0.4330972135066986 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:46,193] Trial 533 finished with value: 0.4664473533630371 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:49,122] Trial 534 finished with value: 0.4549919068813324 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:49,147] Trial 535 finished with value: 0.42484551668167114 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:50:53,559] Trial 536 finished with value: 0.4735533595085144 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:01,806] Trial 539 finished with value: 0.46546807885169983 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:11,677] Trial 537 finished with value: 0.4778256416320801 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:15,318] Trial 538 finished with value: 0.41864681243896484 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:17,981] Trial 540 finished with value: 0.4248729348182678 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:23,062] Trial 541 finished with value: 0.47580328583717346 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:36,646] Trial 542 finished with value: 0.44402050971984863 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:42,170] Trial 543 finished with value: 0.4165906310081482 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:43,910] Trial 544 finished with value: 0.43585866689682007 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:51:46,180] Trial 545 finished with value: 0.4422372877597809 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:02,448] Trial 546 finished with value: 0.43397173285484314 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:09,704] Trial 548 finished with value: 0.4533694386482239 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:09,964] Trial 549 finished with value: 0.41875824332237244 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:10,069] Trial 547 finished with value: 0.4397997260093689 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:24,644] Trial 550 finished with value: 0.4297727644443512 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:34,156] Trial 553 finished with value: 0.46783143281936646 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:37,498] Trial 552 finished with value: 0.44768819212913513 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:38,102] Trial 551 finished with value: 0.43614330887794495 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:44,170] Trial 554 finished with value: 0.44195276498794556 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:52,922] Trial 556 finished with value: 0.47119611501693726 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:53,037] Trial 557 finished with value: 0.45637649297714233 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:52:57,628] Trial 555 finished with value: 0.4343074858188629 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:04,043] Trial 558 finished with value: 0.4150392413139343 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:16,656] Trial 559 finished with value: 0.4572599530220032 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:16,700] Trial 560 finished with value: 0.4481106698513031 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:19,692] Trial 561 finished with value: 0.4228675365447998 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:24,282] Trial 562 finished with value: 0.5027554631233215 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:39,957] Trial 564 finished with value: 0.4410336911678314 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:42,412] Trial 563 finished with value: 0.439260333776474 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:42,671] Trial 566 finished with value: 0.44146594405174255 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:53:42,694] Trial 565 finished with value: 0.43973416090011597 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:01,630] Trial 567 finished with value: 0.5117698907852173 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:04,894] Trial 569 finished with value: 0.4509289860725403 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:04,968] Trial 568 finished with value: 0.45266446471214294 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:05,032] Trial 570 finished with value: 0.4464770257472992 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:25,904] Trial 571 finished with value: 0.532196581363678 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:29,466] Trial 572 finished with value: 0.4689916670322418 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:30,743] Trial 574 finished with value: 0.46798500418663025 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:30,917] Trial 573 finished with value: 0.4749566912651062 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:47,220] Trial 575 finished with value: 0.41335949301719666 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:52,866] Trial 577 finished with value: 0.4297475814819336 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:52,879] Trial 578 finished with value: 0.4347476363182068 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:54:55,022] Trial 576 finished with value: 0.43347907066345215 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:05,003] Trial 579 finished with value: 0.4475630819797516 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:17,489] Trial 582 finished with value: 0.4313139319419861 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:17,562] Trial 580 finished with value: 0.42725706100463867 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:17,600] Trial 581 finished with value: 0.4078783392906189 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:24,536] Trial 583 finished with value: 0.43863487243652344 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:42,801] Trial 584 finished with value: 0.49863362312316895 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:44,896] Trial 585 finished with value: 0.43332090973854065 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:44,919] Trial 586 finished with value: 1.41745126247406 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:55:47,394] Trial 587 finished with value: 0.43572837114334106 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:06,252] Trial 589 finished with value: 0.43787235021591187 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:07,225] Trial 591 finished with value: 0.43813997507095337 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:09,063] Trial 590 finished with value: 0.43658122420310974 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:10,342] Trial 588 finished with value: 1.6520049571990967 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:27,718] Trial 594 finished with value: 0.5467487573623657 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:29,376] Trial 595 finished with value: 0.5757218599319458 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:31,155] Trial 593 finished with value: 0.43533778190612793 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:31,759] Trial 592 finished with value: 0.4479212760925293 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:51,024] Trial 597 finished with value: 0.464876651763916 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:55,784] Trial 596 finished with value: 0.49785447120666504 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:57,925] Trial 599 finished with value: 0.47953540086746216 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:56:57,938] Trial 598 finished with value: 0.4766976833343506 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:00,510] Trial 600 finished with value: 0.46325626969337463 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:09,028] Trial 602 finished with value: 0.5960888266563416 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:09,125] Trial 603 finished with value: 0.5645605325698853 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:18,513] Trial 601 finished with value: 0.5781332850456238 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:26,495] Trial 604 finished with value: 0.6000558733940125 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:33,778] Trial 606 finished with value: 0.42370256781578064 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:33,796] Trial 605 finished with value: 0.5317299365997314 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:40,740] Trial 607 finished with value: 0.4443666338920593 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:45,833] Trial 608 finished with value: 0.4310610890388489 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:57,749] Trial 609 finished with value: 0.42124995589256287 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:57:57,829] Trial 610 finished with value: 0.44655999541282654 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:01,910] Trial 611 finished with value: 0.44686222076416016 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:06,555] Trial 612 finished with value: 0.47169703245162964 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:23,681] Trial 613 finished with value: 0.48001760244369507 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:24,729] Trial 614 finished with value: 0.41751131415367126 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:26,799] Trial 615 finished with value: 0.4350169599056244 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:28,488] Trial 616 finished with value: 0.4487544894218445 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:44,104] Trial 618 finished with value: 0.47228536009788513 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:44,935] Trial 619 finished with value: 0.4453255534172058 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:48,999] Trial 620 finished with value: 0.47753652930259705 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:58:51,039] Trial 617 finished with value: 0.4873073697090149 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:10,907] Trial 621 finished with value: 0.4370288848876953 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:13,572] Trial 622 finished with value: 0.46184101700782776 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:13,725] Trial 623 finished with value: 0.43756571412086487 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:15,467] Trial 624 finished with value: 0.4651831090450287 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:35,476] Trial 628 finished with value: 0.46130865812301636 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:36,057] Trial 627 finished with value: 0.49095726013183594 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:37,982] Trial 625 finished with value: 0.431273877620697 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:38,389] Trial 626 finished with value: 0.4470058083534241 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 16:59:59,466] Trial 629 finished with value: 0.43739914894104004 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:01,976] Trial 630 finished with value: 0.4246732294559479 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:02,182] Trial 632 finished with value: 0.4379311501979828 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:03,542] Trial 631 finished with value: 0.45513391494750977 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:26,981] Trial 633 finished with value: 0.44658517837524414 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:29,563] Trial 635 finished with value: 0.44469112157821655 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:29,621] Trial 636 finished with value: 0.4487380087375641 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:29,649] Trial 634 finished with value: 0.5033689737319946 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:48,612] Trial 637 finished with value: 0.459310382604599 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:52,687] Trial 638 finished with value: 0.4389701783657074 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:52,759] Trial 639 finished with value: 0.4197656512260437 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:00:54,724] Trial 640 finished with value: 0.4243789315223694 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:12,460] Trial 641 finished with value: 0.4444003701210022 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:17,674] Trial 644 finished with value: 0.44400104880332947 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:17,743] Trial 642 finished with value: 0.4066583514213562 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:18,795] Trial 643 finished with value: 0.43158483505249023 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:24,428] Trial 645 finished with value: 0.5293172597885132 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:31,400] Trial 647 finished with value: 0.43620914220809937 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:45,612] Trial 646 finished with value: 0.45410090684890747 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:48,073] Trial 648 finished with value: 0.43419593572616577 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:49,612] Trial 649 finished with value: 0.41486525535583496 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:01:57,413] Trial 650 finished with value: 1.2965877056121826 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:10,637] Trial 651 finished with value: 0.4500857889652252 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:15,253] Trial 652 finished with value: 1.1482590436935425 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:15,284] Trial 653 finished with value: 1.2823985815048218 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:18,276] Trial 654 finished with value: 0.472374826669693 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:32,301] Trial 655 finished with value: 0.43757304549217224 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:43,108] Trial 658 finished with value: 0.45485806465148926 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:43,175] Trial 657 finished with value: 0.44786185026168823 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:43,205] Trial 656 finished with value: 0.4834930896759033 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:02:54,361] Trial 659 finished with value: 0.42879706621170044 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:03,151] Trial 661 finished with value: 0.4456346929073334 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:08,535] Trial 660 finished with value: 0.45328211784362793 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:10,232] Trial 662 finished with value: 0.4866205155849457 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:16,568] Trial 663 finished with value: 0.43098706007003784 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:25,112] Trial 664 finished with value: 0.43594807386398315 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:34,151] Trial 665 finished with value: 0.47666043043136597 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:34,272] Trial 666 finished with value: 0.4379095435142517 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:37,737] Trial 667 finished with value: 0.4825344681739807 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:43,963] Trial 668 finished with value: 0.572878360748291 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:58,749] Trial 669 finished with value: 0.5007466673851013 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:03:59,117] Trial 670 finished with value: 0.4682096838951111 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:01,607] Trial 671 finished with value: 0.42279618978500366 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:08,796] Trial 672 finished with value: 0.43819013237953186 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:25,171] Trial 673 finished with value: 0.4388367235660553 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:31,871] Trial 674 finished with value: 0.47352343797683716 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:33,446] Trial 676 finished with value: 0.42261940240859985 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:35,217] Trial 675 finished with value: 0.4575442671775818 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:47,844] Trial 677 finished with value: 0.40595588088035583 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:57,273] Trial 679 finished with value: 0.4312759339809418 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:04:58,172] Trial 680 finished with value: 0.4124623239040375 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:01,594] Trial 678 finished with value: 0.6040682196617126 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:17,148] Trial 681 finished with value: 0.43645256757736206 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:28,826] Trial 683 finished with value: 0.41817086935043335 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:31,237] Trial 684 finished with value: 0.4466803967952728 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:32,677] Trial 682 finished with value: 0.45900511741638184 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:38,253] Trial 685 finished with value: 0.4238767623901367 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:51,540] Trial 688 finished with value: 0.4933875501155853 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:55,218] Trial 687 finished with value: 0.4540341794490814 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:56,855] Trial 686 finished with value: 0.433878093957901 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:05:57,343] Trial 689 finished with value: 0.42828139662742615 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:09,164] Trial 692 finished with value: 0.4657946527004242 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:13,991] Trial 690 finished with value: 0.49961504340171814 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:21,571] Trial 691 finished with value: 0.4435986876487732 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:23,094] Trial 693 finished with value: 0.4607570767402649 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:30,620] Trial 694 finished with value: 0.5477495193481445 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:38,739] Trial 695 finished with value: 0.4696098864078522 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:48,820] Trial 696 finished with value: 0.4842612147331238 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:49,419] Trial 697 finished with value: 0.4949972629547119 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:53,547] Trial 698 finished with value: 0.4723149240016937 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:06:59,797] Trial 699 finished with value: 0.43486982583999634 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:14,124] Trial 701 finished with value: 0.46319031715393066 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:15,859] Trial 700 finished with value: 0.4249677062034607 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:16,609] Trial 702 finished with value: 0.404521644115448 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:24,317] Trial 703 finished with value: 0.42642492055892944 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:42,331] Trial 706 finished with value: 0.4544065594673157 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:46,534] Trial 704 finished with value: 0.4461558759212494 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:46,562] Trial 707 finished with value: 0.4539770185947418 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:48,026] Trial 705 finished with value: 0.4555380940437317 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:07:59,245] Trial 709 finished with value: 0.4841412603855133 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:05,591] Trial 708 finished with value: 0.4260835647583008 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:06,908] Trial 711 finished with value: 0.4464016854763031 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:08,830] Trial 710 finished with value: 0.41603007912635803 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:17,846] Trial 712 finished with value: 0.45045095682144165 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:21,750] Trial 714 finished with value: 0.47070711851119995 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:26,271] Trial 713 finished with value: 0.4314035475254059 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:28,402] Trial 716 finished with value: 0.4735413193702698 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:29,732] Trial 715 finished with value: 0.42521196603775024 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:41,295] Trial 717 finished with value: 0.43728166818618774 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:48,035] Trial 718 finished with value: 0.4417581558227539 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:50,235] Trial 720 finished with value: 0.4286082684993744 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:08:50,282] Trial 719 finished with value: 0.4352741241455078 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:00,503] Trial 721 finished with value: 0.4330933094024658 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:09,183] Trial 722 finished with value: 0.4601410925388336 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:13,260] Trial 723 finished with value: 0.4704015851020813 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:13,277] Trial 724 finished with value: 0.44894924759864807 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 32}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:20,223] Trial 725 finished with value: 0.4225262999534607 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:32,301] Trial 728 finished with value: 0.4441584646701813 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:32,723] Trial 726 finished with value: 0.523955225944519 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:34,958] Trial 727 finished with value: 0.4405736029148102 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:37,243] Trial 729 finished with value: 0.4432908594608307 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:55,332] Trial 730 finished with value: 0.4570647180080414 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:57,877] Trial 731 finished with value: 0.43645691871643066 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:09:59,230] Trial 732 finished with value: 0.43919187784194946 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:01,139] Trial 733 finished with value: 0.44436851143836975 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:22,391] Trial 734 finished with value: 0.43056732416152954 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:23,007] Trial 735 finished with value: 0.4348270297050476 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:24,274] Trial 736 finished with value: 0.4555925726890564 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:25,497] Trial 737 finished with value: 0.467099130153656 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:47,297] Trial 738 finished with value: 0.44764280319213867 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:48,777] Trial 739 finished with value: 0.42361992597579956 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:49,428] Trial 740 finished with value: 0.49506551027297974 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:49,447] Trial 741 finished with value: 0.4811610281467438 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:10:59,667] Trial 742 finished with value: 0.4330314099788666 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:03,650] Trial 744 finished with value: 0.45260196924209595 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:06,304] Trial 745 finished with value: 0.4893633723258972 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:12,788] Trial 743 finished with value: 0.4496742784976959 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:14,014] Trial 746 finished with value: 0.4582194685935974 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:27,010] Trial 747 finished with value: 0.4418059289455414 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:29,895] Trial 748 finished with value: 0.4371110498905182 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:36,278] Trial 749 finished with value: 0.4826582074165344 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:36,347] Trial 750 finished with value: 0.494695246219635 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:48,763] Trial 751 finished with value: 0.7007375359535217 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:11:53,909] Trial 752 finished with value: 0.7032901048660278 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:00,128] Trial 754 finished with value: 0.45042139291763306 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:00,839] Trial 753 finished with value: 0.43058881163597107 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:16,618] Trial 755 finished with value: 0.4120468199253082 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:19,613] Trial 756 finished with value: 0.43577566742897034 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:22,738] Trial 758 finished with value: 0.4539015293121338 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:26,290] Trial 757 finished with value: 0.44586285948753357 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:38,940] Trial 759 finished with value: 0.46918049454689026 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 10}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:44,359] Trial 761 finished with value: 0.4440156817436218 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:47,344] Trial 760 finished with value: 0.5806461572647095 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:55,108] Trial 764 finished with value: 0.44809260964393616 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:56,296] Trial 762 finished with value: 0.6390677094459534 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:12:59,225] Trial 765 finished with value: 0.46527934074401855 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:03,101] Trial 763 finished with value: 0.4977286756038666 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:22,989] Trial 767 finished with value: 0.434347927570343 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:23,133] Trial 768 finished with value: 0.6076722145080566 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:25,791] Trial 769 finished with value: 0.495645672082901 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:27,720] Trial 766 finished with value: 0.4807385802268982 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 256, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:55,488] Trial 770 finished with value: 0.4551655650138855 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:56,906] Trial 772 finished with value: 0.4393330216407776 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:56,982] Trial 771 finished with value: 0.43388351798057556 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:13:58,469] Trial 773 finished with value: 0.4435886740684509 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:19,510] Trial 777 finished with value: 0.42987290024757385 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:19,667] Trial 776 finished with value: 0.41892582178115845 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:21,021] Trial 774 finished with value: 0.42372697591781616 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:22,979] Trial 775 finished with value: 0.4276164472103119 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:46,410] Trial 778 finished with value: 0.44982337951660156 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:47,624] Trial 780 finished with value: 0.4515914022922516 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:52,083] Trial 781 finished with value: 0.4285767078399658 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:14:53,875] Trial 779 finished with value: 0.494436651468277 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:11,797] Trial 782 finished with value: 0.42686566710472107 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:14,888] Trial 785 finished with value: 0.4566771388053894 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:14,901] Trial 784 finished with value: 0.43725481629371643 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:15,725] Trial 783 finished with value: 0.45110321044921875 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:34,385] Trial 786 finished with value: 0.4337007403373718 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:39,059] Trial 788 finished with value: 0.4372080862522125 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:39,825] Trial 787 finished with value: 0.48617544770240784 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:39,834] Trial 789 finished with value: 0.44985756278038025 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:52,193] Trial 790 finished with value: 0.5014941096305847 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:56,790] Trial 791 finished with value: 0.5169517993927002 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:57,830] Trial 793 finished with value: 0.4795953333377838 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:15:58,654] Trial 792 finished with value: 0.4687870144844055 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:16:16,953] Trial 794 finished with value: 0.4431164264678955 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:16:22,215] Trial 795 finished with value: 0.4538246989250183 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:16:22,956] Trial 797 finished with value: 0.4591025710105896 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:16:23,133] Trial 796 finished with value: 0.47129136323928833 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:16:33,453] Trial 798 finished with value: 0.4403549134731293 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 232 with value: 0.39887046813964844. [I 2021-03-10 17:16:34,070] Trial 799 finished with value: 0.4362393915653229 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 232 with value: 0.39887046813964844.
arch_best_params = study_arch.best_params
display(Markdown("#### Least recorded Loss :"))
print(study_arch.best_value)
display(Markdown("#### Best set of parameters to determine the architecture :"))
print(arch_best_params)
hp_score_board = hp_score_board.append({'Phase':'Architecture Selection',
'Best Value':study_arch.best_value,
'Hyperparameters':f'{study_arch.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.39887 | {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256} |
optuna.visualization.plot_parallel_coordinate(study_arch)
optuna.visualization.plot_optimization_history(study_arch)
study_arch.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_arch | params_neuron_l1 | params_neuron_l2 | params_neuron_l3 | params_neuron_l4 | params_neuron_l5 | params_neuron_l6 | params_neuron_l7 | params_num_layers | state | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.773753 | 2021-03-10 16:01:28.940244 | 2021-03-10 16:01:42.609174 | 0 days 00:00:13.668930 | D B A | 32 | 64.0 | 256.0 | NaN | NaN | NaN | NaN | 3 | COMPLETE |
| 1 | 1 | 0.466368 | 2021-03-10 16:01:28.941419 | 2021-03-10 16:01:49.119355 | 0 days 00:00:20.177936 | A B D | 64 | 256.0 | 128.0 | 256.0 | 32.0 | 10.0 | 64.0 | 7 | COMPLETE |
| 2 | 2 | 0.448074 | 2021-03-10 16:01:28.943187 | 2021-03-10 16:01:44.603632 | 0 days 00:00:15.660445 | A D B | 256 | 64.0 | 10.0 | 128.0 | NaN | NaN | NaN | 4 | COMPLETE |
| 3 | 3 | 0.489002 | 2021-03-10 16:01:28.945566 | 2021-03-10 16:01:44.546003 | 0 days 00:00:15.600437 | D A B | 128 | 10.0 | 256.0 | 32.0 | NaN | NaN | NaN | 4 | COMPLETE |
| 4 | 4 | 0.477922 | 2021-03-10 16:01:42.623020 | 2021-03-10 16:02:03.335669 | 0 days 00:00:20.712649 | B D A | 32 | 32.0 | 64.0 | 10.0 | 128.0 | 256.0 | 64.0 | 7 | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 795 | 795 | 0.453825 | 2021-03-10 17:15:56.797577 | 2021-03-10 17:16:22.214880 | 0 days 00:00:25.417303 | D A B | 64 | 10.0 | 128.0 | 1000.0 | 64.0 | 10.0 | 128.0 | 7 | COMPLETE |
| 796 | 796 | 0.471291 | 2021-03-10 17:15:57.836331 | 2021-03-10 17:16:23.133212 | 0 days 00:00:25.296881 | D A B | 64 | 256.0 | 128.0 | 1000.0 | 64.0 | 10.0 | 128.0 | 7 | COMPLETE |
| 797 | 797 | 0.459103 | 2021-03-10 17:15:58.667786 | 2021-03-10 17:16:22.956116 | 0 days 00:00:24.288330 | D A B | 64 | 10.0 | 128.0 | 1000.0 | 64.0 | 10.0 | 128.0 | 7 | COMPLETE |
| 798 | 798 | 0.440355 | 2021-03-10 17:16:16.959243 | 2021-03-10 17:16:33.453013 | 0 days 00:00:16.493770 | D A B | 32 | 10.0 | 128.0 | 1000.0 | 10.0 | 10.0 | 256.0 | 7 | COMPLETE |
| 799 | 799 | 0.436239 | 2021-03-10 17:16:22.227484 | 2021-03-10 17:16:34.070100 | 0 days 00:00:11.842616 | D A B | 32 | 128.0 | 10.0 | 1000.0 | 10.0 | 32.0 | NaN | 6 | COMPLETE |
800 rows × 15 columns
neurons_per_layer =[arch_best_params[f'neuron_l{i}'] for i in range(1,arch_best_params['num_layers']+1)]
arch_best_params['neurons_per_layer'] = neurons_per_layer
for i in range(1,arch_best_params['num_layers']+1):
if f'neuron_l{i}' in arch_best_params.keys():
del arch_best_params[f'neuron_l{i}']
else:
pass
display(Markdown("#### Modified best parameters from architecture :"))
print(arch_best_params)
{'num_layers': 7, 'arch': 'D A B', 'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256]}
def objective_coarse(trial):
global arch_best_params,loss,metric,X_train,X_test,y_train,y_test
epochs = 50
lr = 0.001
drop_rate_per_layer = [trial.suggest_uniform(f'drop_l{layers}',0.1,0.90) for layers in range(1,arch_best_params['num_layers']+1)]
activation = trial.suggest_categorical('activation',['relu','tanh'])
kernel_init = trial.suggest_categorical('kernel_init', ['glorot_normal',
'glorot_uniform',
'normal',
'uniform',
'he_normal',
'he_uniform',
'lecun_normal',
'lecun_uniform'])
opts = 'adam'
if opts == 'sgd':
optimizer = tf.keras.optimizers.SGD(learning_rate=lr)
elif opts == 'adadelta':
optimizer = tf.keras.optimizers.Adadelta(learning_rate=lr)
elif opts == 'adam':
optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
elif opts == 'rmsprop':
optimizer = tf.keras.optimizers.RMSprop(learning_rate=lr)
kwargs = {
# 'batch_size': batch_size,
'drop_rate_per_layer' :drop_rate_per_layer,
'activation':activation,
'n_epochs': epochs,
'kernel_init': kernel_init,
'opts': optimizer,
}
kwargs = {**arch_best_params, **kwargs}
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1,**kwargs)
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[0]
print(X_train.shape,X_test.shape,y_train.shape,y_test.shape)
study_coarse = optuna.create_study(direction='minimize',sampler=optuna.samplers.TPESampler())
study_coarse.optimize(objective_coarse,n_trials=800,n_jobs=-13)
[I 2021-03-10 17:16:34,640] A new study created in memory with name: no-name-ef7a3040-7ee6-4f4b-be10-4d8a26be5de6 [I 2021-03-10 17:17:07,179] Trial 3 finished with value: 0.5729349851608276 and parameters: {'drop_l1': 0.8440678802749894, 'drop_l2': 0.48565999387015935, 'drop_l3': 0.14754749313806528, 'drop_l4': 0.30433565153030384, 'drop_l5': 0.47452995349492744, 'drop_l6': 0.233403300544669, 'drop_l7': 0.3030767443575968, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 3 with value: 0.5729349851608276. [I 2021-03-10 17:17:07,187] Trial 0 finished with value: 0.642469048500061 and parameters: {'drop_l1': 0.19238398877542312, 'drop_l2': 0.15428288158457334, 'drop_l3': 0.3433330951780243, 'drop_l4': 0.6528031534239299, 'drop_l5': 0.8502307524015112, 'drop_l6': 0.21671290045687527, 'drop_l7': 0.36823700539821747, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 3 with value: 0.5729349851608276. [I 2021-03-10 17:17:07,207] Trial 1 finished with value: 0.8369182348251343 and parameters: {'drop_l1': 0.2986927520293202, 'drop_l2': 0.4291726431985988, 'drop_l3': 0.7981450534044536, 'drop_l4': 0.6061059770635809, 'drop_l5': 0.4433155179868401, 'drop_l6': 0.2638631753379934, 'drop_l7': 0.5516024740810205, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 3 with value: 0.5729349851608276. [I 2021-03-10 17:17:07,219] Trial 2 finished with value: 0.6277998685836792 and parameters: {'drop_l1': 0.1403233690579917, 'drop_l2': 0.7227777986773567, 'drop_l3': 0.8619705850583788, 'drop_l4': 0.6342537341826636, 'drop_l5': 0.49303204550226554, 'drop_l6': 0.16150304304812657, 'drop_l7': 0.5416245755402478, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 3 with value: 0.5729349851608276. [I 2021-03-10 17:17:39,333] Trial 7 finished with value: 2.1384358406066895 and parameters: {'drop_l1': 0.28872800561450485, 'drop_l2': 0.7765636160722829, 'drop_l3': 0.6259192816743908, 'drop_l4': 0.48343177643200785, 'drop_l5': 0.6206038187735687, 'drop_l6': 0.3538218061285783, 'drop_l7': 0.8223020015521731, 'activation': 'tanh', 'kernel_init': 'glorot_normal'}. Best is trial 3 with value: 0.5729349851608276. [I 2021-03-10 17:17:39,344] Trial 6 finished with value: 0.527328610420227 and parameters: {'drop_l1': 0.1033263475912265, 'drop_l2': 0.2190694852587921, 'drop_l3': 0.10723572057500973, 'drop_l4': 0.2300613886846332, 'drop_l5': 0.4950442040377544, 'drop_l6': 0.1732442634604549, 'drop_l7': 0.558445808832741, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:17:39,383] Trial 5 finished with value: 0.5977991819381714 and parameters: {'drop_l1': 0.8949735760255914, 'drop_l2': 0.44537517644905844, 'drop_l3': 0.6574075773354332, 'drop_l4': 0.6659915561810906, 'drop_l5': 0.12648448337009643, 'drop_l6': 0.27697615633046613, 'drop_l7': 0.2622181411749678, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:17:39,390] Trial 4 finished with value: 0.6657489538192749 and parameters: {'drop_l1': 0.4508537483361945, 'drop_l2': 0.8869249272944887, 'drop_l3': 0.8715561597355344, 'drop_l4': 0.6894658179824666, 'drop_l5': 0.12168162252682668, 'drop_l6': 0.4836090552754868, 'drop_l7': 0.3980935680383937, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:18:09,803] Trial 8 finished with value: 0.7859401702880859 and parameters: {'drop_l1': 0.44336345398672183, 'drop_l2': 0.8340171662780523, 'drop_l3': 0.8961761822398366, 'drop_l4': 0.6117448726699691, 'drop_l5': 0.6265325541943884, 'drop_l6': 0.7033143349234848, 'drop_l7': 0.2781138716057787, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:18:12,044] Trial 11 finished with value: 0.5397312641143799 and parameters: {'drop_l1': 0.24672588560940467, 'drop_l2': 0.5060744155817136, 'drop_l3': 0.6691430651509441, 'drop_l4': 0.1828124544474962, 'drop_l5': 0.41274166466912465, 'drop_l6': 0.35131317165011955, 'drop_l7': 0.1290833432523133, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:18:12,047] Trial 10 finished with value: 1.0024688243865967 and parameters: {'drop_l1': 0.301698555911719, 'drop_l2': 0.1547660760184101, 'drop_l3': 0.25205937814266177, 'drop_l4': 0.43497552004462847, 'drop_l5': 0.4093866117764765, 'drop_l6': 0.21602630900348993, 'drop_l7': 0.6728474136803625, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:18:12,048] Trial 9 finished with value: 0.6153764724731445 and parameters: {'drop_l1': 0.6781230048333933, 'drop_l2': 0.46172682682088906, 'drop_l3': 0.8084874715256927, 'drop_l4': 0.1655471689138147, 'drop_l5': 0.21079025352932954, 'drop_l6': 0.4278197033074965, 'drop_l7': 0.8729321916106736, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:18:41,436] Trial 12 finished with value: 0.6548255085945129 and parameters: {'drop_l1': 0.6086134990491909, 'drop_l2': 0.8818779619996143, 'drop_l3': 0.4884795385054186, 'drop_l4': 0.7206120861931488, 'drop_l5': 0.534046852991352, 'drop_l6': 0.7858877001589271, 'drop_l7': 0.4682180281110082, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.527328610420227. [I 2021-03-10 17:18:45,770] Trial 13 finished with value: 0.5156335234642029 and parameters: {'drop_l1': 0.11197850587461627, 'drop_l2': 0.6262075875131226, 'drop_l3': 0.49550624248457154, 'drop_l4': 0.1274506639476196, 'drop_l5': 0.2879925722173282, 'drop_l6': 0.6255117788752551, 'drop_l7': 0.12129139611074494, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 13 with value: 0.5156335234642029. [I 2021-03-10 17:18:45,807] Trial 14 finished with value: 0.5684112310409546 and parameters: {'drop_l1': 0.11884374646879861, 'drop_l2': 0.6191932177290136, 'drop_l3': 0.4846712463284562, 'drop_l4': 0.10247285972141038, 'drop_l5': 0.3032820828041563, 'drop_l6': 0.6050635995176306, 'drop_l7': 0.13287846985575194, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 13 with value: 0.5156335234642029. [I 2021-03-10 17:18:45,829] Trial 15 finished with value: 0.5113908052444458 and parameters: {'drop_l1': 0.10400594225003384, 'drop_l2': 0.29289311142964314, 'drop_l3': 0.5001827948054243, 'drop_l4': 0.11184445226742984, 'drop_l5': 0.3081929048101685, 'drop_l6': 0.6436161128259537, 'drop_l7': 0.15366530602168343, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 15 with value: 0.5113908052444458. [I 2021-03-10 17:19:11,433] Trial 16 finished with value: 0.4891469478607178 and parameters: {'drop_l1': 0.10810688732428261, 'drop_l2': 0.2939201511501855, 'drop_l3': 0.521984932815922, 'drop_l4': 0.10437657674296313, 'drop_l5': 0.30472246973758826, 'drop_l6': 0.10223339261780658, 'drop_l7': 0.1605427794160057, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:19,518] Trial 18 finished with value: 0.6016022562980652 and parameters: {'drop_l1': 0.11506186389842266, 'drop_l2': 0.29267699516577206, 'drop_l3': 0.4179267536842609, 'drop_l4': 0.34596405589512125, 'drop_l5': 0.28145827770670184, 'drop_l6': 0.8992344830860612, 'drop_l7': 0.179842209098449, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:19,520] Trial 19 finished with value: 0.5798527598381042 and parameters: {'drop_l1': 0.3818484386842578, 'drop_l2': 0.2941474591802637, 'drop_l3': 0.34451139992756363, 'drop_l4': 0.34116886730647256, 'drop_l5': 0.2929447880948888, 'drop_l6': 0.8881147990396254, 'drop_l7': 0.10973864221246693, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:19,521] Trial 17 finished with value: 0.6207485795021057 and parameters: {'drop_l1': 0.10279154039605627, 'drop_l2': 0.6325474743921248, 'drop_l3': 0.12957827004793682, 'drop_l4': 0.8702757853910013, 'drop_l5': 0.2794076698565132, 'drop_l6': 0.6138637966267543, 'drop_l7': 0.712417720563914, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:36,266] Trial 20 finished with value: 0.5531358122825623 and parameters: {'drop_l1': 0.394460959725782, 'drop_l2': 0.250156603136244, 'drop_l3': 0.37249689198535313, 'drop_l4': 0.890193922895645, 'drop_l5': 0.3062199706059724, 'drop_l6': 0.7969254559759635, 'drop_l7': 0.181705722736913, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:54,418] Trial 23 finished with value: 0.5145726203918457 and parameters: {'drop_l1': 0.5792684877558094, 'drop_l2': 0.3106598436191794, 'drop_l3': 0.552318833671618, 'drop_l4': 0.2282327543609072, 'drop_l5': 0.17565775802038855, 'drop_l6': 0.776168785558369, 'drop_l7': 0.2095562115122317, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:54,434] Trial 21 finished with value: 0.5483208894729614 and parameters: {'drop_l1': 0.5847276770017279, 'drop_l2': 0.25191861327332415, 'drop_l3': 0.6100504172022696, 'drop_l4': 0.26761274894040354, 'drop_l5': 0.183381366072651, 'drop_l6': 0.7802151224773581, 'drop_l7': 0.2076293502545772, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 16 with value: 0.4891469478607178. [I 2021-03-10 17:19:54,438] Trial 22 finished with value: 0.48246246576309204 and parameters: {'drop_l1': 0.5973593706236151, 'drop_l2': 0.3365976694003764, 'drop_l3': 0.6016145306321039, 'drop_l4': 0.2757636625665237, 'drop_l5': 0.19369773290718642, 'drop_l6': 0.7371042692550264, 'drop_l7': 0.20957402625738297, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:20:03,188] Trial 24 finished with value: 0.5069654583930969 and parameters: {'drop_l1': 0.20692196242726152, 'drop_l2': 0.36678442649476595, 'drop_l3': 0.5947698772843591, 'drop_l4': 0.10959307962596168, 'drop_l5': 0.19772147223254377, 'drop_l6': 0.5868086341483565, 'drop_l7': 0.2038303642246333, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:20:28,655] Trial 25 finished with value: 0.5254276394844055 and parameters: {'drop_l1': 0.5767057960501639, 'drop_l2': 0.3583846967089329, 'drop_l3': 0.7366659131907851, 'drop_l4': 0.11217537826065634, 'drop_l5': 0.2148072071972637, 'drop_l6': 0.5582447668064295, 'drop_l7': 0.34210129659831184, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:20:28,657] Trial 26 finished with value: 0.5874267816543579 and parameters: {'drop_l1': 0.7428139617068668, 'drop_l2': 0.37465384278469116, 'drop_l3': 0.7414916665405742, 'drop_l4': 0.10160273484475721, 'drop_l5': 0.37006619832629983, 'drop_l6': 0.6893786037806868, 'drop_l7': 0.3485087172679169, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:20:28,663] Trial 27 finished with value: 0.5180631279945374 and parameters: {'drop_l1': 0.6719186198102768, 'drop_l2': 0.36634572106629215, 'drop_l3': 0.7259782028680778, 'drop_l4': 0.11756019267814283, 'drop_l5': 0.3510059920824936, 'drop_l6': 0.5518079431031272, 'drop_l7': 0.3545628142395467, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:20:30,228] Trial 28 finished with value: 0.4891602098941803 and parameters: {'drop_l1': 0.7383778815016457, 'drop_l2': 0.3848178888073833, 'drop_l3': 0.7270520635933786, 'drop_l4': 0.4024942115909708, 'drop_l5': 0.1130145422970954, 'drop_l6': 0.5365327482572126, 'drop_l7': 0.35337709915829896, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:00,652] Trial 31 finished with value: 0.5516830682754517 and parameters: {'drop_l1': 0.5211564098801249, 'drop_l2': 0.5319759735125269, 'drop_l3': 0.5675440106756058, 'drop_l4': 0.40648054310314913, 'drop_l5': 0.10259897600059034, 'drop_l6': 0.7118388619072377, 'drop_l7': 0.43446449724464264, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:01,469] Trial 32 finished with value: 0.5132812261581421 and parameters: {'drop_l1': 0.7999712429655679, 'drop_l2': 0.12569683454043246, 'drop_l3': 0.5549049552132224, 'drop_l4': 0.4171485472018056, 'drop_l5': 0.10803308957894374, 'drop_l6': 0.4205005799372348, 'drop_l7': 0.4305088450320674, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:01,580] Trial 29 finished with value: 0.5373049378395081 and parameters: {'drop_l1': 0.7134417621906469, 'drop_l2': 0.5405028198171186, 'drop_l3': 0.5504562086074627, 'drop_l4': 0.4157347520348487, 'drop_l5': 0.12268967988406293, 'drop_l6': 0.10261384164814563, 'drop_l7': 0.2489280921167233, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:01,632] Trial 30 finished with value: 0.49141988158226013 and parameters: {'drop_l1': 0.2021579139512808, 'drop_l2': 0.5441897244316604, 'drop_l3': 0.5689085206548433, 'drop_l4': 0.37871764415199693, 'drop_l5': 0.10389011695107427, 'drop_l6': 0.7063361506479582, 'drop_l7': 0.24020815262872894, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:31,586] Trial 33 finished with value: 0.6444641351699829 and parameters: {'drop_l1': 0.7810549714446516, 'drop_l2': 0.11948141458141498, 'drop_l3': 0.6961353162594605, 'drop_l4': 0.5395557864219799, 'drop_l5': 0.79504442185205, 'drop_l6': 0.47307728960140283, 'drop_l7': 0.2573834307463148, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:33,919] Trial 35 finished with value: 0.6294115781784058 and parameters: {'drop_l1': 0.20322267874308889, 'drop_l2': 0.4063052097084933, 'drop_l3': 0.6887742333244994, 'drop_l4': 0.52792561948185, 'drop_l5': 0.8386221690011568, 'drop_l6': 0.5105845474898071, 'drop_l7': 0.30436998402684223, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:33,949] Trial 36 finished with value: 0.56999671459198 and parameters: {'drop_l1': 0.18256012459335202, 'drop_l2': 0.17981201673258762, 'drop_l3': 0.6846646969694307, 'drop_l4': 0.5397506683578107, 'drop_l5': 0.23221022287173704, 'drop_l6': 0.8578477083025899, 'drop_l7': 0.30083952548560317, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:21:33,960] Trial 34 finished with value: 0.5725823044776917 and parameters: {'drop_l1': 0.1920971267623561, 'drop_l2': 0.19286611738885936, 'drop_l3': 0.6895963935751482, 'drop_l4': 0.5346877404064612, 'drop_l5': 0.8369557787829462, 'drop_l6': 0.5078349264833489, 'drop_l7': 0.2484716052306858, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:00,727] Trial 37 finished with value: 0.5410336256027222 and parameters: {'drop_l1': 0.6316618974424982, 'drop_l2': 0.4264775663797725, 'drop_l3': 0.42554445546103425, 'drop_l4': 0.528513610997025, 'drop_l5': 0.16147656394045284, 'drop_l6': 0.847403853086993, 'drop_l7': 0.31521381669023973, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:05,798] Trial 40 finished with value: 0.6541746258735657 and parameters: {'drop_l1': 0.8736639298947348, 'drop_l2': 0.5815859092546931, 'drop_l3': 0.798730860403655, 'drop_l4': 0.35020964961861223, 'drop_l5': 0.1528912570817526, 'drop_l6': 0.6631640679575469, 'drop_l7': 0.3912026020526466, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:05,844] Trial 39 finished with value: 0.6399227380752563 and parameters: {'drop_l1': 0.6327725578197918, 'drop_l2': 0.5682623348794099, 'drop_l3': 0.8001580305887638, 'drop_l4': 0.35074427979834, 'drop_l5': 0.1567841315735632, 'drop_l6': 0.8334227690967881, 'drop_l7': 0.4137724273739731, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:05,871] Trial 38 finished with value: 0.6048879027366638 and parameters: {'drop_l1': 0.626829860357162, 'drop_l2': 0.5690736077037846, 'drop_l3': 0.7942970192189338, 'drop_l4': 0.3592868368321723, 'drop_l5': 0.15779038348936997, 'drop_l6': 0.6718104402112117, 'drop_l7': 0.4003931243188996, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:24,848] Trial 41 finished with value: 0.6658097505569458 and parameters: {'drop_l1': 0.8930484457734746, 'drop_l2': 0.5706846408467228, 'drop_l3': 0.7992192449860342, 'drop_l4': 0.3475609733298115, 'drop_l5': 0.14597506641287852, 'drop_l6': 0.669233019927014, 'drop_l7': 0.39313243500454764, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:38,408] Trial 44 finished with value: 0.5111448764801025 and parameters: {'drop_l1': 0.1628983504406832, 'drop_l2': 0.3507454380054274, 'drop_l3': 0.6075617610166122, 'drop_l4': 0.2936570089244274, 'drop_l5': 0.24064107620687947, 'drop_l6': 0.7425291277129701, 'drop_l7': 0.21518990981114505, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:38,411] Trial 42 finished with value: 0.5038053393363953 and parameters: {'drop_l1': 0.5115068521135125, 'drop_l2': 0.47934878867782316, 'drop_l3': 0.443117891324165, 'drop_l4': 0.29131355907777523, 'drop_l5': 0.2503170566818929, 'drop_l6': 0.7559382604178203, 'drop_l7': 0.22236035267846335, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:38,412] Trial 43 finished with value: 0.5073179602622986 and parameters: {'drop_l1': 0.522112104821251, 'drop_l2': 0.34433330008569624, 'drop_l3': 0.6295084858955379, 'drop_l4': 0.2581800172457322, 'drop_l5': 0.2473405449674605, 'drop_l6': 0.7457215412012801, 'drop_l7': 0.21279425767866586, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:22:49,097] Trial 45 finished with value: 0.5073906779289246 and parameters: {'drop_l1': 0.2750678057280751, 'drop_l2': 0.32556385393468557, 'drop_l3': 0.6126382023640068, 'drop_l4': 0.28029438743810636, 'drop_l5': 0.246866947511865, 'drop_l6': 0.7373860663481085, 'drop_l7': 0.20686351664254932, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:10,674] Trial 47 finished with value: 1.081829309463501 and parameters: {'drop_l1': 0.41924571853767023, 'drop_l2': 0.480110484922891, 'drop_l3': 0.23852786185415875, 'drop_l4': 0.1817960904436825, 'drop_l5': 0.3477635748393559, 'drop_l6': 0.7261353943478845, 'drop_l7': 0.5131663584707074, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:10,675] Trial 48 finished with value: 0.5177348852157593 and parameters: {'drop_l1': 0.4569121874997435, 'drop_l2': 0.47712203789127444, 'drop_l3': 0.44811908487766394, 'drop_l4': 0.1776621940571952, 'drop_l5': 0.10050783421708195, 'drop_l6': 0.3097260532272257, 'drop_l7': 0.15955222853378237, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:10,681] Trial 46 finished with value: 0.9936891794204712 and parameters: {'drop_l1': 0.4651143592606021, 'drop_l2': 0.4526329425613078, 'drop_l3': 0.4425604073606939, 'drop_l4': 0.4656375717669025, 'drop_l5': 0.3595229039242655, 'drop_l6': 0.734221113258809, 'drop_l7': 0.5015907657423126, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:15,464] Trial 49 finished with value: 0.7545366883277893 and parameters: {'drop_l1': 0.48009940410965035, 'drop_l2': 0.4690357976257284, 'drop_l3': 0.29135040223746, 'drop_l4': 0.4807297848531811, 'drop_l5': 0.3488448586841677, 'drop_l6': 0.8125622672513797, 'drop_l7': 0.15972081466610089, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:42,421] Trial 52 finished with value: 0.6110168099403381 and parameters: {'drop_l1': 0.3651584323056162, 'drop_l2': 0.7107169939183863, 'drop_l3': 0.5213073883630903, 'drop_l4': 0.23470933197704147, 'drop_l5': 0.549901307803261, 'drop_l6': 0.823060643862124, 'drop_l7': 0.10246220153397997, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:43,353] Trial 50 finished with value: 0.6502876281738281 and parameters: {'drop_l1': 0.5059226127295984, 'drop_l2': 0.6894831637395902, 'drop_l3': 0.6527992296909614, 'drop_l4': 0.230980096780454, 'drop_l5': 0.5603520298581044, 'drop_l6': 0.8113759219230041, 'drop_l7': 0.10585926995325531, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:43,404] Trial 51 finished with value: 0.596019983291626 and parameters: {'drop_l1': 0.5406346315791789, 'drop_l2': 0.7022060783602204, 'drop_l3': 0.5176355881054442, 'drop_l4': 0.21855743987724974, 'drop_l5': 0.5697438699613715, 'drop_l6': 0.41542635511441023, 'drop_l7': 0.27800974889298424, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:23:44,900] Trial 53 finished with value: 0.5799891352653503 and parameters: {'drop_l1': 0.5353020624584424, 'drop_l2': 0.6950163248558845, 'drop_l3': 0.5341666026641656, 'drop_l4': 0.23250244934797987, 'drop_l5': 0.45025429181072085, 'drop_l6': 0.5453096287282034, 'drop_l7': 0.28285852568276254, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:24:13,190] Trial 54 finished with value: 0.5072250366210938 and parameters: {'drop_l1': 0.3261957033681688, 'drop_l2': 0.39329295995343894, 'drop_l3': 0.6478802140241581, 'drop_l4': 0.3910404827394564, 'drop_l5': 0.19427905659470243, 'drop_l6': 0.5863064711878208, 'drop_l7': 0.2705707666660255, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:24:15,435] Trial 55 finished with value: 0.48314595222473145 and parameters: {'drop_l1': 0.32706076740084356, 'drop_l2': 0.41275112659662955, 'drop_l3': 0.5892936005670345, 'drop_l4': 0.39231727030298486, 'drop_l5': 0.19967383232327118, 'drop_l6': 0.5819884372540056, 'drop_l7': 0.24566729990603048, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:24:15,484] Trial 57 finished with value: 0.5088119506835938 and parameters: {'drop_l1': 0.23315118751876468, 'drop_l2': 0.39726414406176025, 'drop_l3': 0.47667143230870157, 'drop_l4': 0.38553668816711895, 'drop_l5': 0.20268138671585106, 'drop_l6': 0.5840959086336582, 'drop_l7': 0.23606118347539384, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:24:15,539] Trial 56 finished with value: 0.6004752516746521 and parameters: {'drop_l1': 0.3313445568696841, 'drop_l2': 0.394243836378719, 'drop_l3': 0.39073989942702697, 'drop_l4': 0.3814088814415319, 'drop_l5': 0.19959904611287355, 'drop_l6': 0.5821497751458599, 'drop_l7': 0.2415641292084345, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:24:43,655] Trial 58 finished with value: 0.5359058380126953 and parameters: {'drop_l1': 0.2478225426464003, 'drop_l2': 0.2627648410137183, 'drop_l3': 0.5815232307746616, 'drop_l4': 0.15201803067942427, 'drop_l5': 0.20431956835889423, 'drop_l6': 0.5879008379279236, 'drop_l7': 0.23659369058227683, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 22 with value: 0.48246246576309204. [I 2021-03-10 17:24:47,035] Trial 59 finished with value: 0.4779098629951477 and parameters: {'drop_l1': 0.33440702907620146, 'drop_l2': 0.25576866841030577, 'drop_l3': 0.4722399984129307, 'drop_l4': 0.3134935756370124, 'drop_l5': 0.267616520772247, 'drop_l6': 0.6449894984326766, 'drop_l7': 0.6186803402395791, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 59 with value: 0.4779098629951477. [I 2021-03-10 17:24:47,043] Trial 61 finished with value: 0.4488716125488281 and parameters: {'drop_l1': 0.1552233083659346, 'drop_l2': 0.520179446974375, 'drop_l3': 0.5877673268111666, 'drop_l4': 0.3086722968953137, 'drop_l5': 0.2717391054439378, 'drop_l6': 0.4592494253074211, 'drop_l7': 0.14765602973691166, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:24:47,059] Trial 60 finished with value: 0.5193318724632263 and parameters: {'drop_l1': 0.6900129133014414, 'drop_l2': 0.25172389854383626, 'drop_l3': 0.5798491192835343, 'drop_l4': 0.4494124007918545, 'drop_l5': 0.25659440281684653, 'drop_l6': 0.6246867923346354, 'drop_l7': 0.616701787469127, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:25:11,985] Trial 62 finished with value: 0.5320644974708557 and parameters: {'drop_l1': 0.13598633441194632, 'drop_l2': 0.43451857658246607, 'drop_l3': 0.4680958486607128, 'drop_l4': 0.3047719437993873, 'drop_l5': 0.2729954704614377, 'drop_l6': 0.4566756726102671, 'drop_l7': 0.18052946784044882, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:25:18,695] Trial 65 finished with value: 0.5397960543632507 and parameters: {'drop_l1': 0.14699880876489665, 'drop_l2': 0.5189493077913794, 'drop_l3': 0.4664357101210707, 'drop_l4': 0.3276434199177977, 'drop_l5': 0.27700769373251477, 'drop_l6': 0.6475768539532483, 'drop_l7': 0.5825648606552865, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:25:18,699] Trial 64 finished with value: 0.5382509827613831 and parameters: {'drop_l1': 0.14746635191223453, 'drop_l2': 0.4373423392649167, 'drop_l3': 0.47547740560343305, 'drop_l4': 0.4370245143342599, 'drop_l5': 0.31552882247579106, 'drop_l6': 0.4591855651014948, 'drop_l7': 0.7378466820014838, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:25:18,716] Trial 63 finished with value: 0.5851370096206665 and parameters: {'drop_l1': 0.27179172217123065, 'drop_l2': 0.5107986885417254, 'drop_l3': 0.47474797390922124, 'drop_l4': 0.3153968433882997, 'drop_l5': 0.3284610890349079, 'drop_l6': 0.6450239289276877, 'drop_l7': 0.7590721160762384, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:25:36,836] Trial 66 finished with value: 0.5285175442695618 and parameters: {'drop_l1': 0.1576563335763536, 'drop_l2': 0.5126914403978604, 'drop_l3': 0.5039516583905778, 'drop_l4': 0.3091490256237354, 'drop_l5': 0.32266656450197284, 'drop_l6': 0.646746578717471, 'drop_l7': 0.7127641614982482, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 61 with value: 0.4488716125488281. [I 2021-03-10 17:25:53,167] Trial 69 finished with value: 0.4450318217277527 and parameters: {'drop_l1': 0.3149865928079203, 'drop_l2': 0.219387107525433, 'drop_l3': 0.5222361402801331, 'drop_l4': 0.49687771632764843, 'drop_l5': 0.12974189645677442, 'drop_l6': 0.13017252483124822, 'drop_l7': 0.13415524093034925, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 69 with value: 0.4450318217277527. [I 2021-03-10 17:25:53,169] Trial 67 finished with value: 0.4515143036842346 and parameters: {'drop_l1': 0.2702159068141633, 'drop_l2': 0.21147316175236053, 'drop_l3': 0.8466183410555703, 'drop_l4': 0.37765350015008303, 'drop_l5': 0.13206419444398027, 'drop_l6': 0.14423669677533385, 'drop_l7': 0.6525077945282096, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 69 with value: 0.4450318217277527. [I 2021-03-10 17:25:53,185] Trial 68 finished with value: 0.457430899143219 and parameters: {'drop_l1': 0.27156413718423067, 'drop_l2': 0.32437365815208374, 'drop_l3': 0.5182908558887267, 'drop_l4': 0.3688949255573093, 'drop_l5': 0.4142072913763793, 'drop_l6': 0.5350771903299605, 'drop_l7': 0.12774438721411274, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 69 with value: 0.4450318217277527. [I 2021-03-10 17:26:02,904] Trial 70 finished with value: 0.49622997641563416 and parameters: {'drop_l1': 0.23028080951083085, 'drop_l2': 0.20856736696293282, 'drop_l3': 0.7607306176575452, 'drop_l4': 0.37824516367571387, 'drop_l5': 0.1322019261930618, 'drop_l6': 0.5331541431019579, 'drop_l7': 0.13894856404440453, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 69 with value: 0.4450318217277527. [I 2021-03-10 17:26:28,596] Trial 72 finished with value: 0.4781469702720642 and parameters: {'drop_l1': 0.32091791727791297, 'drop_l2': 0.22692910980906508, 'drop_l3': 0.8629457358298691, 'drop_l4': 0.6394103360062218, 'drop_l5': 0.3978170919399535, 'drop_l6': 0.1615239592298926, 'drop_l7': 0.6438546582834838, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 69 with value: 0.4450318217277527. [I 2021-03-10 17:26:28,606] Trial 71 finished with value: 0.45238351821899414 and parameters: {'drop_l1': 0.3220087630754719, 'drop_l2': 0.2733583004910429, 'drop_l3': 0.8858497002264499, 'drop_l4': 0.5899698965227527, 'drop_l5': 0.3899799707790286, 'drop_l6': 0.14987310166550016, 'drop_l7': 0.6389117827834426, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 69 with value: 0.4450318217277527. [I 2021-03-10 17:26:28,614] Trial 73 finished with value: 0.43124693632125854 and parameters: {'drop_l1': 0.30740453300959514, 'drop_l2': 0.22392168259856102, 'drop_l3': 0.893779046922865, 'drop_l4': 0.25584581208204443, 'drop_l5': 0.3991088695205296, 'drop_l6': 0.16231069267803974, 'drop_l7': 0.6577923526441871, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 73 with value: 0.43124693632125854. [I 2021-03-10 17:26:31,700] Trial 74 finished with value: 0.4592515528202057 and parameters: {'drop_l1': 0.34271929204439644, 'drop_l2': 0.22458749272506992, 'drop_l3': 0.8803261821219919, 'drop_l4': 0.26527142051479025, 'drop_l5': 0.4112203715052726, 'drop_l6': 0.10323783615749976, 'drop_l7': 0.6229504464892069, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 73 with value: 0.43124693632125854. [I 2021-03-10 17:27:09,662] Trial 75 finished with value: 0.4693271517753601 and parameters: {'drop_l1': 0.31022276465562604, 'drop_l2': 0.15865415515169737, 'drop_l3': 0.8978099626950529, 'drop_l4': 0.7312025638250583, 'drop_l5': 0.394989329424675, 'drop_l6': 0.15316470062896062, 'drop_l7': 0.6317756721266349, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 73 with value: 0.43124693632125854. [I 2021-03-10 17:27:10,573] Trial 77 finished with value: 0.48771676421165466 and parameters: {'drop_l1': 0.35779921405537973, 'drop_l2': 0.1530657334456737, 'drop_l3': 0.8942513662130385, 'drop_l4': 0.7023599218460761, 'drop_l5': 0.3983135369766362, 'drop_l6': 0.13801637424826482, 'drop_l7': 0.6407005977517253, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 73 with value: 0.43124693632125854. [I 2021-03-10 17:27:12,295] Trial 76 finished with value: 0.47584113478660583 and parameters: {'drop_l1': 0.35667134939035383, 'drop_l2': 0.15436266248814634, 'drop_l3': 0.8994900528505755, 'drop_l4': 0.6958107038850774, 'drop_l5': 0.42586488790935906, 'drop_l6': 0.14369540409778037, 'drop_l7': 0.6276682250655593, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 73 with value: 0.43124693632125854. [I 2021-03-10 17:27:12,437] Trial 78 finished with value: 0.4282829165458679 and parameters: {'drop_l1': 0.35747239182090634, 'drop_l2': 0.1581463532232153, 'drop_l3': 0.8290310408207331, 'drop_l4': 0.6845038050995153, 'drop_l5': 0.47359763010352585, 'drop_l6': 0.19380015446250315, 'drop_l7': 0.6654482645036945, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:27:50,642] Trial 79 finished with value: 0.4345524311065674 and parameters: {'drop_l1': 0.36448858366227627, 'drop_l2': 0.16606107774053333, 'drop_l3': 0.8385183202804829, 'drop_l4': 0.7836406324235112, 'drop_l5': 0.4675691045062616, 'drop_l6': 0.12906192812596423, 'drop_l7': 0.6751553829209404, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:27:52,895] Trial 82 finished with value: 0.45948782563209534 and parameters: {'drop_l1': 0.2944823495141148, 'drop_l2': 0.22654295433961838, 'drop_l3': 0.8369508757886115, 'drop_l4': 0.5647912201956893, 'drop_l5': 0.504947048756157, 'drop_l6': 0.20802987358794872, 'drop_l7': 0.668131746183691, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:27:53,002] Trial 80 finished with value: 0.46365851163864136 and parameters: {'drop_l1': 0.29462553324842533, 'drop_l2': 0.15886843895134406, 'drop_l3': 0.8334616471416849, 'drop_l4': 0.7714782607489036, 'drop_l5': 0.4362376133596769, 'drop_l6': 0.19651524612914428, 'drop_l7': 0.6755281794929944, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:27:53,052] Trial 81 finished with value: 0.4478282928466797 and parameters: {'drop_l1': 0.29795849200184854, 'drop_l2': 0.17526477110250893, 'drop_l3': 0.8405908040257012, 'drop_l4': 0.8044342028978931, 'drop_l5': 0.50529592190184, 'drop_l6': 0.22705029326252615, 'drop_l7': 0.5721304583316816, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:28:30,269] Trial 83 finished with value: 0.5201148986816406 and parameters: {'drop_l1': 0.29618346251588495, 'drop_l2': 0.2777021249208754, 'drop_l3': 0.8388815529831876, 'drop_l4': 0.7528745094218595, 'drop_l5': 0.49403239510512414, 'drop_l6': 0.2216420203133534, 'drop_l7': 0.6676690759810049, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:28:33,161] Trial 86 finished with value: 0.4715006947517395 and parameters: {'drop_l1': 0.4086251773671556, 'drop_l2': 0.1881625102482562, 'drop_l3': 0.8347476429131594, 'drop_l4': 0.7748227910991349, 'drop_l5': 0.4718247700494198, 'drop_l6': 0.23628871342842045, 'drop_l7': 0.5769188397622185, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:28:34,062] Trial 85 finished with value: 0.48130273818969727 and parameters: {'drop_l1': 0.39952514236993203, 'drop_l2': 0.27520352518601554, 'drop_l3': 0.8389769016721451, 'drop_l4': 0.5997936569669258, 'drop_l5': 0.46638523993061926, 'drop_l6': 0.24563747871183955, 'drop_l7': 0.5795033285990244, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:28:34,186] Trial 84 finished with value: 0.5096259713172913 and parameters: {'drop_l1': 0.40517157142819604, 'drop_l2': 0.18291053229834195, 'drop_l3': 0.8292780591392853, 'drop_l4': 0.7438679841014861, 'drop_l5': 0.4684712804116694, 'drop_l6': 0.190888781100092, 'drop_l7': 0.5924505570603358, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:11,407] Trial 87 finished with value: 0.43251556158065796 and parameters: {'drop_l1': 0.42508163143022265, 'drop_l2': 0.10173189474322226, 'drop_l3': 0.7707264674206092, 'drop_l4': 0.8205453530929888, 'drop_l5': 0.4762281143729501, 'drop_l6': 0.2528060158312303, 'drop_l7': 0.5690756369868917, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:15,505] Trial 90 finished with value: 0.44486474990844727 and parameters: {'drop_l1': 0.26105551061633603, 'drop_l2': 0.10809633861099435, 'drop_l3': 0.768357171427706, 'drop_l4': 0.8170080853369251, 'drop_l5': 0.5968754048861721, 'drop_l6': 0.12415459054017788, 'drop_l7': 0.6872721986629372, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:15,525] Trial 89 finished with value: 0.46810752153396606 and parameters: {'drop_l1': 0.27107423945187514, 'drop_l2': 0.10327101704291564, 'drop_l3': 0.7781908826025157, 'drop_l4': 0.8196449307415818, 'drop_l5': 0.5873510005506244, 'drop_l6': 0.12143621630526297, 'drop_l7': 0.530901760536608, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:15,543] Trial 88 finished with value: 0.5167363286018372 and parameters: {'drop_l1': 0.3893377217378878, 'drop_l2': 0.13546821787729726, 'drop_l3': 0.7738518845134655, 'drop_l4': 0.824195422553101, 'drop_l5': 0.5141997039735013, 'drop_l6': 0.2598065432738147, 'drop_l7': 0.7857335388515241, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:48,977] Trial 91 finished with value: 0.49291959404945374 and parameters: {'drop_l1': 0.43985230426419675, 'drop_l2': 0.1060273238520355, 'drop_l3': 0.7779763792114115, 'drop_l4': 0.8188193127256292, 'drop_l5': 0.5279906516755177, 'drop_l6': 0.1204811303017506, 'drop_l7': 0.8064155396480166, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:58,645] Trial 92 finished with value: 0.5464086532592773 and parameters: {'drop_l1': 0.37916475282095924, 'drop_l2': 0.1058948569883223, 'drop_l3': 0.7638846051657041, 'drop_l4': 0.8105925943928335, 'drop_l5': 0.6528437210882647, 'drop_l6': 0.1768047806420319, 'drop_l7': 0.6974825241386057, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:58,749] Trial 93 finished with value: 0.6192471385002136 and parameters: {'drop_l1': 0.3812215769992412, 'drop_l2': 0.10418523230867166, 'drop_l3': 0.8737267831407841, 'drop_l4': 0.8434831194030549, 'drop_l5': 0.722864008780741, 'drop_l6': 0.1770065521552349, 'drop_l7': 0.7019236604375696, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:29:58,777] Trial 94 finished with value: 0.5278269648551941 and parameters: {'drop_l1': 0.3735769004328393, 'drop_l2': 0.10434876029279244, 'drop_l3': 0.8753023617879808, 'drop_l4': 0.8641496124795027, 'drop_l5': 0.6242523351312105, 'drop_l6': 0.1804022693617015, 'drop_l7': 0.7092963317433014, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:30:19,871] Trial 95 finished with value: 0.4995555877685547 and parameters: {'drop_l1': 0.37873418309248424, 'drop_l2': 0.1343576157475902, 'drop_l3': 0.8729712050310555, 'drop_l4': 0.8694407163891577, 'drop_l5': 0.4455999423009633, 'drop_l6': 0.18601948870742013, 'drop_l7': 0.6931484832020245, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:30:41,017] Trial 98 finished with value: 0.49212098121643066 and parameters: {'drop_l1': 0.2507360882524275, 'drop_l2': 0.1362096655442836, 'drop_l3': 0.853061939414367, 'drop_l4': 0.6636814389964674, 'drop_l5': 0.3809622853954263, 'drop_l6': 0.28757530069502996, 'drop_l7': 0.65344730465782, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:30:41,152] Trial 96 finished with value: 0.5428317189216614 and parameters: {'drop_l1': 0.2507267643645277, 'drop_l2': 0.20180947141728245, 'drop_l3': 0.8705870981749406, 'drop_l4': 0.8563232442137052, 'drop_l5': 0.6785902760129525, 'drop_l6': 0.12351775889881567, 'drop_l7': 0.5600272521612802, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:30:41,177] Trial 97 finished with value: 0.4971176087856293 and parameters: {'drop_l1': 0.24525309705575626, 'drop_l2': 0.2041552250933249, 'drop_l3': 0.8190950313470835, 'drop_l4': 0.8910277109779909, 'drop_l5': 0.5974738524565425, 'drop_l6': 0.1274199522623875, 'drop_l7': 0.5440289794203037, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:30:50,720] Trial 99 finished with value: 0.6065041422843933 and parameters: {'drop_l1': 0.43052655136101775, 'drop_l2': 0.20362911513115622, 'drop_l3': 0.8166247321198077, 'drop_l4': 0.8996123444169021, 'drop_l5': 0.5976685551025017, 'drop_l6': 0.30106987997574586, 'drop_l7': 0.47811718739339804, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:31:21,900] Trial 101 finished with value: 0.5843721628189087 and parameters: {'drop_l1': 0.437734896301004, 'drop_l2': 0.16497700529084958, 'drop_l3': 0.7121111485756074, 'drop_l4': 0.8005835483705006, 'drop_l5': 0.48964011464129287, 'drop_l6': 0.34545020720777875, 'drop_l7': 0.6001056484593786, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:31:22,713] Trial 102 finished with value: 0.5149883031845093 and parameters: {'drop_l1': 0.21625091014440434, 'drop_l2': 0.16721259747413353, 'drop_l3': 0.7133019620985558, 'drop_l4': 0.5779984659261814, 'drop_l5': 0.523946524657597, 'drop_l6': 0.10251709362562426, 'drop_l7': 0.4756662677350821, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 78 with value: 0.4282829165458679. [I 2021-03-10 17:31:23,201] Trial 100 finished with value: 0.4207402765750885 and parameters: {'drop_l1': 0.3508181167413091, 'drop_l2': 0.20215327755581042, 'drop_l3': 0.8181591374600683, 'drop_l4': 0.7883656824982926, 'drop_l5': 0.5900874758086131, 'drop_l6': 0.12399621775861508, 'drop_l7': 0.5364403646360837, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 100 with value: 0.4207402765750885. [I 2021-03-10 17:31:26,156] Trial 103 finished with value: 0.5632425546646118 and parameters: {'drop_l1': 0.2199776171184623, 'drop_l2': 0.17222962656725477, 'drop_l3': 0.7059135334750866, 'drop_l4': 0.7933456900394817, 'drop_l5': 0.5354117273173724, 'drop_l6': 0.36206693019470654, 'drop_l7': 0.7385888141735848, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 100 with value: 0.4207402765750885. [I 2021-03-10 17:32:03,290] Trial 104 finished with value: 0.47090038657188416 and parameters: {'drop_l1': 0.34701284153272616, 'drop_l2': 0.23678333608638044, 'drop_l3': 0.8503157734967309, 'drop_l4': 0.6026376889114213, 'drop_l5': 0.42276783425058545, 'drop_l6': 0.10042235305742629, 'drop_l7': 0.7306859376821215, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 100 with value: 0.4207402765750885. [I 2021-03-10 17:32:04,311] Trial 107 finished with value: 0.439375102519989 and parameters: {'drop_l1': 0.34731921416923683, 'drop_l2': 0.24089710889153446, 'drop_l3': 0.7918716598146713, 'drop_l4': 0.8369692808160298, 'drop_l5': 0.44880004750649843, 'drop_l6': 0.16475488415037826, 'drop_l7': 0.6048153698202388, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 100 with value: 0.4207402765750885. [I 2021-03-10 17:32:04,334] Trial 105 finished with value: 0.410471111536026 and parameters: {'drop_l1': 0.31195920186521686, 'drop_l2': 0.31977456708535595, 'drop_l3': 0.8584190540132939, 'drop_l4': 0.5018474490617308, 'drop_l5': 0.422558869426746, 'drop_l6': 0.1568387300271355, 'drop_l7': 0.7467774488464978, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:32:04,354] Trial 106 finished with value: 0.4334515631198883 and parameters: {'drop_l1': 0.3445010247165128, 'drop_l2': 0.1764589546473055, 'drop_l3': 0.8047930369721263, 'drop_l4': 0.7900633903526547, 'drop_l5': 0.5429368861939783, 'drop_l6': 0.158915207818476, 'drop_l7': 0.7416386452573244, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:32:45,828] Trial 108 finished with value: 0.4276478886604309 and parameters: {'drop_l1': 0.30898607667145117, 'drop_l2': 0.12317395355602703, 'drop_l3': 0.8084449150236311, 'drop_l4': 0.8389263111954022, 'drop_l5': 0.547485049593347, 'drop_l6': 0.15782000806265967, 'drop_l7': 0.5610535640006952, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:32:46,476] Trial 111 finished with value: 0.4540039598941803 and parameters: {'drop_l1': 0.3157161482679869, 'drop_l2': 0.240768192804997, 'drop_l3': 0.8101624691968059, 'drop_l4': 0.7795005802792208, 'drop_l5': 0.563643525169946, 'drop_l6': 0.16347691715225704, 'drop_l7': 0.777646518423276, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:32:48,526] Trial 109 finished with value: 0.6100246906280518 and parameters: {'drop_l1': 0.308312509800941, 'drop_l2': 0.24106227591236334, 'drop_l3': 0.7566129307492581, 'drop_l4': 0.8359488827440383, 'drop_l5': 0.5459067068798558, 'drop_l6': 0.16435391741045738, 'drop_l7': 0.8564950996777345, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:32:48,943] Trial 110 finished with value: 0.43669503927230835 and parameters: {'drop_l1': 0.31104599043226244, 'drop_l2': 0.12146551449499232, 'drop_l3': 0.7496860187750337, 'drop_l4': 0.7739955996320155, 'drop_l5': 0.547617062494028, 'drop_l6': 0.16318481503877064, 'drop_l7': 0.7680330036871907, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:33:28,940] Trial 112 finished with value: 1.6246989965438843 and parameters: {'drop_l1': 0.3126589193165511, 'drop_l2': 0.12330967588631342, 'drop_l3': 0.746624538608527, 'drop_l4': 0.8427500730634682, 'drop_l5': 0.5545246108030093, 'drop_l6': 0.1640713883916556, 'drop_l7': 0.8542751180460134, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:33:30,849] Trial 113 finished with value: 3.9099650382995605 and parameters: {'drop_l1': 0.3462273503133887, 'drop_l2': 0.12151560766531971, 'drop_l3': 0.7464722836254812, 'drop_l4': 0.8392319034977487, 'drop_l5': 0.6444430829896587, 'drop_l6': 0.20706626634445174, 'drop_l7': 0.8981072505930541, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:33:31,665] Trial 114 finished with value: 0.42754417657852173 and parameters: {'drop_l1': 0.34818899171479245, 'drop_l2': 0.12786351970091156, 'drop_l3': 0.7856457066547831, 'drop_l4': 0.8440252372766852, 'drop_l5': 0.4896353065904878, 'drop_l6': 0.21129725711226693, 'drop_l7': 0.5569682304202617, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:33:31,672] Trial 115 finished with value: 0.48448657989501953 and parameters: {'drop_l1': 0.33858637331287383, 'drop_l2': 0.12061404210804472, 'drop_l3': 0.7453692946000832, 'drop_l4': 0.75188666894619, 'drop_l5': 0.5874301516017508, 'drop_l6': 0.20532784666177306, 'drop_l7': 0.6045867637670779, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:10,988] Trial 116 finished with value: 0.46571141481399536 and parameters: {'drop_l1': 0.34721952982456317, 'drop_l2': 0.14560919867449718, 'drop_l3': 0.7900230933538198, 'drop_l4': 0.7832839763956888, 'drop_l5': 0.48320348769494653, 'drop_l6': 0.20754929318181548, 'drop_l7': 0.7575231096083243, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:12,523] Trial 117 finished with value: 0.4604140818119049 and parameters: {'drop_l1': 0.3648768008731881, 'drop_l2': 0.1394134701180148, 'drop_l3': 0.7871458618164475, 'drop_l4': 0.7631421619955578, 'drop_l5': 0.5875531474660814, 'drop_l6': 0.13422936144849018, 'drop_l7': 0.5236127690005234, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:12,981] Trial 118 finished with value: 0.4725554883480072 and parameters: {'drop_l1': 0.34129306810889926, 'drop_l2': 0.13779706310520337, 'drop_l3': 0.7890785125749361, 'drop_l4': 0.8835366628458534, 'drop_l5': 0.4560677964550911, 'drop_l6': 0.259059810912229, 'drop_l7': 0.5115224336883603, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:13,798] Trial 119 finished with value: 0.43164125084877014 and parameters: {'drop_l1': 0.3630972999546865, 'drop_l2': 0.14095998093846598, 'drop_l3': 0.7883975913323199, 'drop_l4': 0.7195638560926297, 'drop_l5': 0.48437778520022123, 'drop_l6': 0.24516492195271214, 'drop_l7': 0.5227537714714969, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:51,217] Trial 120 finished with value: 0.4426818788051605 and parameters: {'drop_l1': 0.3659404175652916, 'drop_l2': 0.18996370696570802, 'drop_l3': 0.7842623592237267, 'drop_l4': 0.8812902223394361, 'drop_l5': 0.4562150056434873, 'drop_l6': 0.1154297536238012, 'drop_l7': 0.5268402291648426, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:51,910] Trial 121 finished with value: 0.46483224630355835 and parameters: {'drop_l1': 0.48107419108681454, 'drop_l2': 0.18833147878212525, 'drop_l3': 0.8177045259874105, 'drop_l4': 0.7242218426729775, 'drop_l5': 0.4493083013277559, 'drop_l6': 0.2579510807965887, 'drop_l7': 0.5559936772220966, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:53,686] Trial 122 finished with value: 0.5015653371810913 and parameters: {'drop_l1': 0.41657285432787705, 'drop_l2': 0.18435658558843146, 'drop_l3': 0.8173442930422778, 'drop_l4': 0.7225155495631708, 'drop_l5': 0.4345968712146629, 'drop_l6': 0.11092205244757215, 'drop_l7': 0.6799717666016806, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:34:53,992] Trial 123 finished with value: 0.5221966505050659 and parameters: {'drop_l1': 0.4156730806470059, 'drop_l2': 0.1893461905911411, 'drop_l3': 0.811761713689681, 'drop_l4': 0.7216518216953941, 'drop_l5': 0.4345998125690425, 'drop_l6': 0.2366577001251235, 'drop_l7': 0.5527461044120106, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:35:32,058] Trial 124 finished with value: 0.46378788352012634 and parameters: {'drop_l1': 0.35991679238735086, 'drop_l2': 0.18276238049009963, 'drop_l3': 0.8171402595240251, 'drop_l4': 0.7346790035058525, 'drop_l5': 0.4389811889288232, 'drop_l6': 0.24228220297081704, 'drop_l7': 0.555099369416085, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:35:33,040] Trial 125 finished with value: 0.4860248565673828 and parameters: {'drop_l1': 0.4141068873829199, 'drop_l2': 0.1860340134415719, 'drop_l3': 0.8035558093130925, 'drop_l4': 0.6792246820994929, 'drop_l5': 0.4848467642272613, 'drop_l6': 0.24165298611287678, 'drop_l7': 0.4950039010416352, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:35:34,855] Trial 127 finished with value: 0.4816029965877533 and parameters: {'drop_l1': 0.3593292626657915, 'drop_l2': 0.16910628104739842, 'drop_l3': 0.8004701580514569, 'drop_l4': 0.6817844177214358, 'drop_l5': 0.5085023013131698, 'drop_l6': 0.1508121643859159, 'drop_l7': 0.500276255230912, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:35:34,936] Trial 126 finished with value: 0.43118199706077576 and parameters: {'drop_l1': 0.3675268364635234, 'drop_l2': 0.15055997287078907, 'drop_l3': 0.7309152615332544, 'drop_l4': 0.8551336262758167, 'drop_l5': 0.48741773411031813, 'drop_l6': 0.23916313266445516, 'drop_l7': 0.5438166377011883, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:11,589] Trial 128 finished with value: 0.4499078691005707 and parameters: {'drop_l1': 0.2876450935833822, 'drop_l2': 0.3026991010002545, 'drop_l3': 0.7987334497562248, 'drop_l4': 0.7922480420815117, 'drop_l5': 0.5079476417784823, 'drop_l6': 0.1547602298312067, 'drop_l7': 0.49165857937037777, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:14,470] Trial 129 finished with value: 0.48289600014686584 and parameters: {'drop_l1': 0.29010934246098424, 'drop_l2': 0.30642473388841773, 'drop_l3': 0.7354809473312547, 'drop_l4': 0.7875036614967376, 'drop_l5': 0.520527100282543, 'drop_l6': 0.19189722616455496, 'drop_l7': 0.5289536039407059, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:16,766] Trial 130 finished with value: 0.4582788944244385 and parameters: {'drop_l1': 0.28837656031184083, 'drop_l2': 0.14862518936970331, 'drop_l3': 0.8602736380511496, 'drop_l4': 0.7966274738232451, 'drop_l5': 0.4660743109639934, 'drop_l6': 0.22116889150726832, 'drop_l7': 0.4441053774539772, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:16,823] Trial 131 finished with value: 0.48314014077186584 and parameters: {'drop_l1': 0.27874322501023535, 'drop_l2': 0.14317131767054142, 'drop_l3': 0.7274420413874376, 'drop_l4': 0.7980225637161281, 'drop_l5': 0.5212589917857966, 'drop_l6': 0.21710593886743684, 'drop_l7': 0.612623063788229, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:48,930] Trial 132 finished with value: 0.4957645833492279 and parameters: {'drop_l1': 0.3944738125316663, 'drop_l2': 0.15264222089302523, 'drop_l3': 0.7286860354377862, 'drop_l4': 0.8551938450660702, 'drop_l5': 0.5419361253577923, 'drop_l6': 0.2733744137796484, 'drop_l7': 0.6081124085707874, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:54,651] Trial 133 finished with value: 0.49325743317604065 and parameters: {'drop_l1': 0.3903326364842665, 'drop_l2': 0.15561897504377914, 'drop_l3': 0.7279669687147963, 'drop_l4': 0.706094492741248, 'drop_l5': 0.4792337612792831, 'drop_l6': 0.21983044890646158, 'drop_l7': 0.8057067373135866, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:56,719] Trial 135 finished with value: 0.4434072971343994 and parameters: {'drop_l1': 0.38335645001369145, 'drop_l2': 0.2141768273344446, 'drop_l3': 0.7634951631671305, 'drop_l4': 0.8509119129881445, 'drop_l5': 0.48208652096115256, 'drop_l6': 0.17336222689458528, 'drop_l7': 0.5905960719970814, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:36:57,088] Trial 134 finished with value: 0.47638291120529175 and parameters: {'drop_l1': 0.39629196981684683, 'drop_l2': 0.15587918252682736, 'drop_l3': 0.7768287949024392, 'drop_l4': 0.8548148135174818, 'drop_l5': 0.5435852600021307, 'drop_l6': 0.2805736482820898, 'drop_l7': 0.537497194626287, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:37:21,178] Trial 136 finished with value: 0.48711878061294556 and parameters: {'drop_l1': 0.328036929766242, 'drop_l2': 0.21348347911715407, 'drop_l3': 0.759201944862254, 'drop_l4': 0.8289006993909342, 'drop_l5': 0.4758138956538264, 'drop_l6': 0.17452500619369604, 'drop_l7': 0.5705317811839443, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:37:34,200] Trial 137 finished with value: 0.4717136025428772 and parameters: {'drop_l1': 0.32967398246248963, 'drop_l2': 0.11974498216135167, 'drop_l3': 0.7752577656766075, 'drop_l4': 0.8289147560421952, 'drop_l5': 0.4594038387142689, 'drop_l6': 0.17566236813402797, 'drop_l7': 0.5380273994029018, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:37:36,539] Trial 138 finished with value: 0.45776551961898804 and parameters: {'drop_l1': 0.3328464313586079, 'drop_l2': 0.12036723981448523, 'drop_l3': 0.6782566096231986, 'drop_l4': 0.8283533920689002, 'drop_l5': 0.4582408305317538, 'drop_l6': 0.1839172856244371, 'drop_l7': 0.5677418483141642, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:37:36,690] Trial 139 finished with value: 0.6194791793823242 and parameters: {'drop_l1': 0.33573256276655955, 'drop_l2': 0.1215739392454418, 'drop_l3': 0.7843733622820638, 'drop_l4': 0.8780976351211712, 'drop_l5': 0.4510204765028877, 'drop_l6': 0.18509019646394176, 'drop_l7': 0.5700270433726741, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:37:52,291] Trial 140 finished with value: 0.6344199776649475 and parameters: {'drop_l1': 0.3674208956094423, 'drop_l2': 0.12614921855735395, 'drop_l3': 0.8546331699295158, 'drop_l4': 0.8721645136695396, 'drop_l5': 0.4568581783721451, 'drop_l6': 0.19469375590330695, 'drop_l7': 0.7274040599353044, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:14,442] Trial 141 finished with value: 0.5433764457702637 and parameters: {'drop_l1': 0.3702005611233239, 'drop_l2': 0.19676926001199335, 'drop_l3': 0.6788215953115355, 'drop_l4': 0.8784147538668929, 'drop_l5': 0.4244651972481345, 'drop_l6': 0.19371341299853273, 'drop_l7': 0.45367278911293407, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:17,049] Trial 143 finished with value: 0.6279133558273315 and parameters: {'drop_l1': 0.37198274935038467, 'drop_l2': 0.20011738921089808, 'drop_l3': 0.8301213569884022, 'drop_l4': 0.7637407068301187, 'drop_l5': 0.5731193539731394, 'drop_l6': 0.139574258362347, 'drop_l7': 0.7227241496181145, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:17,126] Trial 142 finished with value: 0.5386874079704285 and parameters: {'drop_l1': 0.3664232386974652, 'drop_l2': 0.2625281623154526, 'drop_l3': 0.8532499592780595, 'drop_l4': 0.7614913862774862, 'drop_l5': 0.41868313909765115, 'drop_l6': 0.1469089096635024, 'drop_l7': 0.7247951225867214, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:25,007] Trial 144 finished with value: 2.2126877307891846 and parameters: {'drop_l1': 0.36722137153976797, 'drop_l2': 0.20015216556506424, 'drop_l3': 0.8267888382832116, 'drop_l4': 0.8994462650327916, 'drop_l5': 0.5681055594720662, 'drop_l6': 0.14605184200438018, 'drop_l7': 0.7570970766996636, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:54,213] Trial 145 finished with value: 0.4489031732082367 and parameters: {'drop_l1': 0.30520881515180687, 'drop_l2': 0.21843773921580997, 'drop_l3': 0.8283109741459767, 'drop_l4': 0.7557918838854469, 'drop_l5': 0.4953731387168085, 'drop_l6': 0.14191999555861912, 'drop_l7': 0.5924945918010893, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:59,748] Trial 147 finished with value: 0.46876055002212524 and parameters: {'drop_l1': 0.35052141672358417, 'drop_l2': 0.22366344967887103, 'drop_l3': 0.7572352570086262, 'drop_l4': 0.8128003560776687, 'drop_l5': 0.4907344940606789, 'drop_l6': 0.11403504594484273, 'drop_l7': 0.7526301996517342, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:38:59,749] Trial 146 finished with value: 0.45852574706077576 and parameters: {'drop_l1': 0.3525862169932219, 'drop_l2': 0.22700010803204274, 'drop_l3': 0.7606183275341944, 'drop_l4': 0.8971616750808058, 'drop_l5': 0.487523366651646, 'drop_l6': 0.11021772098926251, 'drop_l7': 0.5971242449790737, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:39:01,632] Trial 148 finished with value: 0.47208231687545776 and parameters: {'drop_l1': 0.34668344170564563, 'drop_l2': 0.23048444209037888, 'drop_l3': 0.7621688546427361, 'drop_l4': 0.8117144070860149, 'drop_l5': 0.4961865921913672, 'drop_l6': 0.11198347079777077, 'drop_l7': 0.5823814997282214, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:39:29,582] Trial 149 finished with value: 0.4376339912414551 and parameters: {'drop_l1': 0.3547922086313126, 'drop_l2': 0.1714804337900411, 'drop_l3': 0.7608855977196249, 'drop_l4': 0.6293504405340419, 'drop_l5': 0.36625489925146965, 'drop_l6': 0.11544863177492112, 'drop_l7': 0.5138971094645478, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:39:40,649] Trial 151 finished with value: 0.43397650122642517 and parameters: {'drop_l1': 0.4275017203521788, 'drop_l2': 0.1745173008290841, 'drop_l3': 0.7991024363725583, 'drop_l4': 0.7416589914313485, 'drop_l5': 0.534345315757043, 'drop_l6': 0.16565177286318755, 'drop_l7': 0.5077096332735997, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:39:42,681] Trial 152 finished with value: 0.4844140112400055 and parameters: {'drop_l1': 0.3889471512748891, 'drop_l2': 0.17243813999191607, 'drop_l3': 0.7965414389600378, 'drop_l4': 0.633254878135638, 'drop_l5': 0.5319534543363221, 'drop_l6': 0.16609533307876256, 'drop_l7': 0.6561043207810052, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:39:43,117] Trial 150 finished with value: 0.48969539999961853 and parameters: {'drop_l1': 0.3890711020967575, 'drop_l2': 0.16790630797839703, 'drop_l3': 0.7939120730379118, 'drop_l4': 0.8552343557855214, 'drop_l5': 0.40306117443052997, 'drop_l6': 0.16194530598692744, 'drop_l7': 0.5143370004295799, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:40:02,349] Trial 153 finished with value: 0.43255338072776794 and parameters: {'drop_l1': 0.31965425220436366, 'drop_l2': 0.16534732722644896, 'drop_l3': 0.801500509304409, 'drop_l4': 0.6441981456518754, 'drop_l5': 0.36171275659229096, 'drop_l6': 0.16241257942906587, 'drop_l7': 0.5194147519826245, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:40:22,055] Trial 154 finished with value: 0.43536314368247986 and parameters: {'drop_l1': 0.4671844259709635, 'drop_l2': 0.16634470343126315, 'drop_l3': 0.7931747469598196, 'drop_l4': 0.6374211519301212, 'drop_l5': 0.3681118830454176, 'drop_l6': 0.16495854607523727, 'drop_l7': 0.5077683178637754, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:40:23,842] Trial 155 finished with value: 0.43252143263816833 and parameters: {'drop_l1': 0.465225969443106, 'drop_l2': 0.1692743538782189, 'drop_l3': 0.8047999909144307, 'drop_l4': 0.7428467317198175, 'drop_l5': 0.37450608034141697, 'drop_l6': 0.13219350008472755, 'drop_l7': 0.5160971412230579, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:40:23,873] Trial 156 finished with value: 0.44532322883605957 and parameters: {'drop_l1': 0.46326654637048764, 'drop_l2': 0.10080095788700945, 'drop_l3': 0.8043937514897097, 'drop_l4': 0.7414370205609355, 'drop_l5': 0.3580898464291313, 'drop_l6': 0.13217544639668616, 'drop_l7': 0.5203391637079924, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 105 with value: 0.410471111536026. [I 2021-03-10 17:40:36,005] Trial 157 finished with value: 0.4069921374320984 and parameters: {'drop_l1': 0.46440547672558896, 'drop_l2': 0.14132368686304508, 'drop_l3': 0.8890208522401337, 'drop_l4': 0.6475286838241638, 'drop_l5': 0.36255781351859595, 'drop_l6': 0.20550746142496662, 'drop_l7': 0.4830086162523807, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:01,716] Trial 158 finished with value: 0.44249433279037476 and parameters: {'drop_l1': 0.4597588235811998, 'drop_l2': 0.1403681994890194, 'drop_l3': 0.8085533423373492, 'drop_l4': 0.6513974306387826, 'drop_l5': 0.3725101809639494, 'drop_l6': 0.13098809931794017, 'drop_l7': 0.4801830654539334, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:04,945] Trial 159 finished with value: 0.4609962999820709 and parameters: {'drop_l1': 0.46897572865595466, 'drop_l2': 0.13835783432118257, 'drop_l3': 0.8860475195437694, 'drop_l4': 0.6504253652787512, 'drop_l5': 0.34062822551040484, 'drop_l6': 0.20472327151269223, 'drop_l7': 0.4540561567585537, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:04,961] Trial 160 finished with value: 0.44124236702919006 and parameters: {'drop_l1': 0.4504008103540389, 'drop_l2': 0.15211242956894747, 'drop_l3': 0.8962007889437964, 'drop_l4': 0.6702077321485012, 'drop_l5': 0.3819076899247779, 'drop_l6': 0.2069014234841689, 'drop_l7': 0.4818311828331997, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:09,326] Trial 161 finished with value: 0.4250360131263733 and parameters: {'drop_l1': 0.4716209167972472, 'drop_l2': 0.1400973155797383, 'drop_l3': 0.8904557693512625, 'drop_l4': 0.6534122860134219, 'drop_l5': 0.34352572808750276, 'drop_l6': 0.20292786138460164, 'drop_l7': 0.48423821831955394, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:38,367] Trial 162 finished with value: 0.4643748700618744 and parameters: {'drop_l1': 0.4200852132794964, 'drop_l2': 0.15851767056880883, 'drop_l3': 0.8910648403445152, 'drop_l4': 0.7114374820860763, 'drop_l5': 0.3288905449525634, 'drop_l6': 0.21123714784633457, 'drop_l7': 0.46076260469601504, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:45,159] Trial 163 finished with value: 0.45269936323165894 and parameters: {'drop_l1': 0.446739859309106, 'drop_l2': 0.10320104253655993, 'drop_l3': 0.8412229441483217, 'drop_l4': 0.7083188948159234, 'drop_l5': 0.3253000949016867, 'drop_l6': 0.22494856969567267, 'drop_l7': 0.5456063502847177, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:46,198] Trial 164 finished with value: 0.5136752724647522 and parameters: {'drop_l1': 0.49508014984809656, 'drop_l2': 0.10183860731065952, 'drop_l3': 0.8406649883499022, 'drop_l4': 0.7043391566951313, 'drop_l5': 0.3806781159790328, 'drop_l6': 0.22882850195527354, 'drop_l7': 0.7800595707679404, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:41:47,472] Trial 165 finished with value: 0.4572319984436035 and parameters: {'drop_l1': 0.4894701358369859, 'drop_l2': 0.10040636557308763, 'drop_l3': 0.8658434171026694, 'drop_l4': 0.6983812496667564, 'drop_l5': 0.3350923800772169, 'drop_l6': 0.22525312090728325, 'drop_l7': 0.4651394656774355, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:42:09,690] Trial 166 finished with value: 0.5343008637428284 and parameters: {'drop_l1': 0.4361821839763503, 'drop_l2': 0.17081113256955247, 'drop_l3': 0.8622778498246962, 'drop_l4': 0.6940190912137836, 'drop_l5': 0.3009348942930462, 'drop_l6': 0.24518510954248265, 'drop_l7': 0.49526527944142607, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:42:25,781] Trial 167 finished with value: 0.4385179877281189 and parameters: {'drop_l1': 0.49941769391504226, 'drop_l2': 0.16987221942554864, 'drop_l3': 0.8651082124946204, 'drop_l4': 0.6171596569251893, 'drop_l5': 0.37726012851443247, 'drop_l6': 0.19461270981969353, 'drop_l7': 0.5017194676463458, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:42:27,239] Trial 168 finished with value: 0.5252994298934937 and parameters: {'drop_l1': 0.4891356544801564, 'drop_l2': 0.1713487458868485, 'drop_l3': 0.8689853842164981, 'drop_l4': 0.6868066397683208, 'drop_l5': 0.34319064726864007, 'drop_l6': 0.19559560288682204, 'drop_l7': 0.4237069233311265, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:42:27,663] Trial 169 finished with value: 0.4969085156917572 and parameters: {'drop_l1': 0.5574227728356816, 'drop_l2': 0.17036543349798158, 'drop_l3': 0.8577859424480003, 'drop_l4': 0.6099596000201167, 'drop_l5': 0.3922277147429896, 'drop_l6': 0.18949843264817834, 'drop_l7': 0.498743835708194, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:42:41,348] Trial 170 finished with value: 0.4505590498447418 and parameters: {'drop_l1': 0.5525041981342406, 'drop_l2': 0.13200382041761527, 'drop_l3': 0.8232658915828472, 'drop_l4': 0.620893663564637, 'drop_l5': 0.4036663244235552, 'drop_l6': 0.1941234579463613, 'drop_l7': 0.4163719059573602, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:07,122] Trial 171 finished with value: 0.5182842016220093 and parameters: {'drop_l1': 0.5536745403136274, 'drop_l2': 0.13532549490931156, 'drop_l3': 0.8209769942636498, 'drop_l4': 0.662276596935952, 'drop_l5': 0.3531357244339032, 'drop_l6': 0.1526662670751482, 'drop_l7': 0.430603553746605, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:09,247] Trial 173 finished with value: 0.5786961913108826 and parameters: {'drop_l1': 0.514698214712374, 'drop_l2': 0.14320579837595732, 'drop_l3': 0.822648596690774, 'drop_l4': 0.7373276425026976, 'drop_l5': 0.3627677125572072, 'drop_l6': 0.15142082475029428, 'drop_l7': 0.5422787226421812, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:09,381] Trial 172 finished with value: 0.4506116509437561 and parameters: {'drop_l1': 0.42180747136245794, 'drop_l2': 0.13461055498396857, 'drop_l3': 0.8206346049974401, 'drop_l4': 0.6600564413724236, 'drop_l5': 0.3559199457722856, 'drop_l6': 0.15315864404392385, 'drop_l7': 0.5458686041689581, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:15,629] Trial 174 finished with value: 0.4819714426994324 and parameters: {'drop_l1': 0.47585981292488516, 'drop_l2': 0.146904903743169, 'drop_l3': 0.8068688921005877, 'drop_l4': 0.6528537462461144, 'drop_l5': 0.6102579927557543, 'drop_l6': 0.1587056535163982, 'drop_l7': 0.5317015739249698, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:48,329] Trial 175 finished with value: 0.48072606325149536 and parameters: {'drop_l1': 0.43165602407048814, 'drop_l2': 0.11676991965672623, 'drop_l3': 0.7814385293012912, 'drop_l4': 0.645659410559663, 'drop_l5': 0.36327047197505835, 'drop_l6': 0.16851923484308654, 'drop_l7': 0.5381472599699786, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:51,217] Trial 177 finished with value: 0.4713863432407379 and parameters: {'drop_l1': 0.3132554837465582, 'drop_l2': 0.11424760657007424, 'drop_l3': 0.7743446600747057, 'drop_l4': 0.7694603429695877, 'drop_l5': 0.5193993006868252, 'drop_l6': 0.1635080523043399, 'drop_l7': 0.5130264908981713, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:51,948] Trial 176 finished with value: 0.505948543548584 and parameters: {'drop_l1': 0.4710847527731266, 'drop_l2': 0.11255388557888374, 'drop_l3': 0.8996131939237544, 'drop_l4': 0.7845421851188416, 'drop_l5': 0.5178783071736013, 'drop_l6': 0.17051429016568842, 'drop_l7': 0.5193028818599121, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:43:53,256] Trial 178 finished with value: 0.49962320923805237 and parameters: {'drop_l1': 0.3102079655497245, 'drop_l2': 0.8447432569225843, 'drop_l3': 0.7853709450788299, 'drop_l4': 0.7727486881756344, 'drop_l5': 0.5137021390535459, 'drop_l6': 0.1765230455697306, 'drop_l7': 0.5111415991591238, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:44:27,004] Trial 179 finished with value: 0.4676702916622162 and parameters: {'drop_l1': 0.3136565261143838, 'drop_l2': 0.18602005164268084, 'drop_l3': 0.8444109747154558, 'drop_l4': 0.7748566032547424, 'drop_l5': 0.5201136619246115, 'drop_l6': 0.17453105622849044, 'drop_l7': 0.5091314209139447, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:44:30,949] Trial 180 finished with value: 0.46885767579078674 and parameters: {'drop_l1': 0.32412028363768997, 'drop_l2': 0.18539704065334095, 'drop_l3': 0.8402486843128822, 'drop_l4': 0.784539712318594, 'drop_l5': 0.5591993808272334, 'drop_l6': 0.17525623549186292, 'drop_l7': 0.48256256112603557, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:44:31,039] Trial 181 finished with value: 0.8992022275924683 and parameters: {'drop_l1': 0.32079356815232773, 'drop_l2': 0.1838114827126311, 'drop_l3': 0.8424491750870132, 'drop_l4': 0.5103236760118781, 'drop_l5': 0.5575408104446945, 'drop_l6': 0.1380721942495897, 'drop_l7': 0.47552261065226104, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:44:32,390] Trial 182 finished with value: 0.4931563436985016 and parameters: {'drop_l1': 0.4526821113963748, 'drop_l2': 0.18691809213559524, 'drop_l3': 0.8427764551601485, 'drop_l4': 0.5739716172072035, 'drop_l5': 0.5540943491963329, 'drop_l6': 0.2535552469964014, 'drop_l7': 0.4787426965948001, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:03,640] Trial 183 finished with value: 0.5387448072433472 and parameters: {'drop_l1': 0.32557932641722237, 'drop_l2': 0.15161624515756478, 'drop_l3': 0.7438635640242636, 'drop_l4': 0.7492620038676909, 'drop_l5': 0.5523019340347789, 'drop_l6': 0.24847513203101385, 'drop_l7': 0.5601367808913925, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:11,826] Trial 185 finished with value: 0.4804263710975647 and parameters: {'drop_l1': 0.40399221049246653, 'drop_l2': 0.15903179128972078, 'drop_l3': 0.7451373383906417, 'drop_l4': 0.5771307887061117, 'drop_l5': 0.41152425958340505, 'drop_l6': 0.13002339559900522, 'drop_l7': 0.5655112516628418, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:11,832] Trial 184 finished with value: 0.49177616834640503 and parameters: {'drop_l1': 0.32684312068837473, 'drop_l2': 0.15242130794055217, 'drop_l3': 0.745798891514615, 'drop_l4': 0.6349034271038927, 'drop_l5': 0.4121816059867671, 'drop_l6': 0.100791908354766, 'drop_l7': 0.5645553380043393, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:11,915] Trial 186 finished with value: 0.4773043692111969 and parameters: {'drop_l1': 0.40316909400785145, 'drop_l2': 0.15960958935676717, 'drop_l3': 0.7448408251603411, 'drop_l4': 0.6270869858931297, 'drop_l5': 0.36876176593412396, 'drop_l6': 0.12097937911687177, 'drop_l7': 0.7930832261367673, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:35,035] Trial 187 finished with value: 0.4307074546813965 and parameters: {'drop_l1': 0.30166028813895757, 'drop_l2': 0.16244029323021048, 'drop_l3': 0.8051139985979626, 'drop_l4': 0.6767239348491293, 'drop_l5': 0.311475280831189, 'drop_l6': 0.12861382677533, 'drop_l7': 0.49373249479387926, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:55,078] Trial 190 finished with value: 0.5061317682266235 and parameters: {'drop_l1': 0.3432016489375891, 'drop_l2': 0.20649505824199813, 'drop_l3': 0.7894962050160418, 'drop_l4': 0.686149800196667, 'drop_l5': 0.3077623063766356, 'drop_l6': 0.27184117020851273, 'drop_l7': 0.7444984285820802, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:55,101] Trial 189 finished with value: 0.4156564772129059 and parameters: {'drop_l1': 0.2968749836661049, 'drop_l2': 0.1280187596467805, 'drop_l3': 0.8048254899556533, 'drop_l4': 0.6701018198220053, 'drop_l5': 0.31592578067271115, 'drop_l6': 0.13986798468128403, 'drop_l7': 0.5268731414319086, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:45:55,119] Trial 188 finished with value: 0.4368739128112793 and parameters: {'drop_l1': 0.2937969504696701, 'drop_l2': 0.12602843375632078, 'drop_l3': 0.7981408700305908, 'drop_l4': 0.6253917834431997, 'drop_l5': 0.37336671381099273, 'drop_l6': 0.27196733468415524, 'drop_l7': 0.527274860712339, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:46:08,537] Trial 191 finished with value: 0.5403696298599243 and parameters: {'drop_l1': 0.2957988361862758, 'drop_l2': 0.1280197343390261, 'drop_l3': 0.8036384571264612, 'drop_l4': 0.6711152768605252, 'drop_l5': 0.5801955002294712, 'drop_l6': 0.26758127674244797, 'drop_l7': 0.766233593542517, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:46:40,090] Trial 193 finished with value: 0.48343315720558167 and parameters: {'drop_l1': 0.2783049410363574, 'drop_l2': 0.13078392206583003, 'drop_l3': 0.8004504358382744, 'drop_l4': 0.6710993506454995, 'drop_l5': 0.28600252327605835, 'drop_l6': 0.1457869097649835, 'drop_l7': 0.49675054020861664, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:46:40,128] Trial 194 finished with value: 0.44248780608177185 and parameters: {'drop_l1': 0.27949037474242255, 'drop_l2': 0.1278439344212033, 'drop_l3': 0.807351375956622, 'drop_l4': 0.8064666593521249, 'drop_l5': 0.3221990644689933, 'drop_l6': 0.13854260140341154, 'drop_l7': 0.49517809457893447, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:46:40,158] Trial 192 finished with value: 0.4904734492301941 and parameters: {'drop_l1': 0.29819800303148203, 'drop_l2': 0.1275893415857437, 'drop_l3': 0.8075559203523847, 'drop_l4': 0.6706415548984882, 'drop_l5': 0.28459979735659785, 'drop_l6': 0.14251009619589788, 'drop_l7': 0.7691634323475509, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:46:45,252] Trial 195 finished with value: 0.4991808533668518 and parameters: {'drop_l1': 0.2783568890357289, 'drop_l2': 0.14363926098071697, 'drop_l3': 0.8106656052187908, 'drop_l4': 0.6755016117558461, 'drop_l5': 0.3148078557624562, 'drop_l6': 0.1379278401923631, 'drop_l7': 0.8266799531720664, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:47:24,364] Trial 198 finished with value: 0.43968313932418823 and parameters: {'drop_l1': 0.45637422819103607, 'drop_l2': 0.16028083231334392, 'drop_l3': 0.8780492432130079, 'drop_l4': 0.7208578259098206, 'drop_l5': 0.3382194032814408, 'drop_l6': 0.21125976535491434, 'drop_l7': 0.5296916542582004, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:47:25,638] Trial 199 finished with value: 0.44853097200393677 and parameters: {'drop_l1': 0.30671946244081794, 'drop_l2': 0.16388278304275525, 'drop_l3': 0.8828399307687808, 'drop_l4': 0.7243991181648125, 'drop_l5': 0.33728169402539365, 'drop_l6': 0.20485813331137004, 'drop_l7': 0.5262899929584082, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:47:25,659] Trial 196 finished with value: 0.46661263704299927 and parameters: {'drop_l1': 0.30496834823384333, 'drop_l2': 0.14643631658763737, 'drop_l3': 0.776473255285107, 'drop_l4': 0.7234881947326005, 'drop_l5': 0.3112579217808938, 'drop_l6': 0.1016087775506096, 'drop_l7': 0.5475333916294923, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:47:25,683] Trial 197 finished with value: 0.4132031500339508 and parameters: {'drop_l1': 0.30653093372239887, 'drop_l2': 0.1510003352500583, 'drop_l3': 0.7737951447505138, 'drop_l4': 0.7198341034844448, 'drop_l5': 0.5008274914142754, 'drop_l6': 0.2088959146280952, 'drop_l7': 0.5264568359890464, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:07,529] Trial 200 finished with value: 0.5204890966415405 and parameters: {'drop_l1': 0.3026874342210095, 'drop_l2': 0.19891653961559497, 'drop_l3': 0.770175056382641, 'drop_l4': 0.7339729760083546, 'drop_l5': 0.5311214605435839, 'drop_l6': 0.10012459138478665, 'drop_l7': 0.5471683843918524, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:09,865] Trial 201 finished with value: 0.5117955803871155 and parameters: {'drop_l1': 0.3334936758517721, 'drop_l2': 0.20210616895298456, 'drop_l3': 0.826940569566311, 'drop_l4': 0.739820089222741, 'drop_l5': 0.5046521308082658, 'drop_l6': 0.23294717193767467, 'drop_l7': 0.5079208022721413, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:09,910] Trial 202 finished with value: 0.5957630276679993 and parameters: {'drop_l1': 0.43833295459554883, 'drop_l2': 0.1729581428818799, 'drop_l3': 0.8275049024040192, 'drop_l4': 0.7430681816500208, 'drop_l5': 0.47871224845776905, 'drop_l6': 0.18564356679582075, 'drop_l7': 0.44772397671897557, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:10,285] Trial 203 finished with value: 0.5439245700836182 and parameters: {'drop_l1': 0.34189219590197645, 'drop_l2': 0.20578138403530594, 'drop_l3': 0.8263445262859954, 'drop_l4': 0.7072487550545236, 'drop_l5': 0.49920626887004294, 'drop_l6': 0.187191518449944, 'drop_l7': 0.5087463429385599, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:50,209] Trial 204 finished with value: 0.48520904779434204 and parameters: {'drop_l1': 0.33963254612769167, 'drop_l2': 0.17452278770930035, 'drop_l3': 0.7879483270344593, 'drop_l4': 0.6904992925827949, 'drop_l5': 0.5074821552800686, 'drop_l6': 0.18847532746265622, 'drop_l7': 0.490812887868091, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:55,268] Trial 205 finished with value: 0.4351643919944763 and parameters: {'drop_l1': 0.26464055085333005, 'drop_l2': 0.17529130312461844, 'drop_l3': 0.7862164649722844, 'drop_l4': 0.6915204663912404, 'drop_l5': 0.4993610683589643, 'drop_l6': 0.1589078605143945, 'drop_l7': 0.48807565177703827, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:55,321] Trial 206 finished with value: 0.44000014662742615 and parameters: {'drop_l1': 0.2582194800240855, 'drop_l2': 0.11278518138445993, 'drop_l3': 0.7864108841537026, 'drop_l4': 0.7025771857169231, 'drop_l5': 0.5005800879744251, 'drop_l6': 0.15851982028222653, 'drop_l7': 0.48910596995369116, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:48:55,356] Trial 207 finished with value: 0.47459760308265686 and parameters: {'drop_l1': 0.35730351579076064, 'drop_l2': 0.1499779851635671, 'drop_l3': 0.7871504010988574, 'drop_l4': 0.6935761358829727, 'drop_l5': 0.5401324281209902, 'drop_l6': 0.21803851066271127, 'drop_l7': 0.6677849439542974, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:49:27,335] Trial 208 finished with value: 0.4292075037956238 and parameters: {'drop_l1': 0.3227186184373888, 'drop_l2': 0.11534964395452438, 'drop_l3': 0.17972781685516076, 'drop_l4': 0.7572023596302496, 'drop_l5': 0.4711697888294534, 'drop_l6': 0.15488921439073555, 'drop_l7': 0.6652729531820617, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:49:39,251] Trial 209 finished with value: 0.4244714677333832 and parameters: {'drop_l1': 0.2558514153447871, 'drop_l2': 0.14283075343790066, 'drop_l3': 0.7723897203739579, 'drop_l4': 0.6449264116395298, 'drop_l5': 0.4742255687293847, 'drop_l6': 0.11891494925888936, 'drop_l7': 0.5304082295728397, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:49:39,992] Trial 211 finished with value: 0.45334386825561523 and parameters: {'drop_l1': 0.26408466195509367, 'drop_l2': 0.17823433664046634, 'drop_l3': 0.773718595784299, 'drop_l4': 0.6514894499020831, 'drop_l5': 0.48241283294368914, 'drop_l6': 0.15542660969048241, 'drop_l7': 0.5250980650355666, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:49:40,075] Trial 210 finished with value: 0.4187901020050049 and parameters: {'drop_l1': 0.26718136246327656, 'drop_l2': 0.15073261049407236, 'drop_l3': 0.7689040068816053, 'drop_l4': 0.656891749494066, 'drop_l5': 0.4802201485339732, 'drop_l6': 0.12303186932685609, 'drop_l7': 0.4706546653285286, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:49:57,074] Trial 212 finished with value: 0.4461396634578705 and parameters: {'drop_l1': 0.23195698699013026, 'drop_l2': 0.14541539055539382, 'drop_l3': 0.17010802948335277, 'drop_l4': 0.5530313351132073, 'drop_l5': 0.4708751649385255, 'drop_l6': 0.1263074254804507, 'drop_l7': 0.6892627437849108, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:50:21,009] Trial 215 finished with value: 0.44998884201049805 and parameters: {'drop_l1': 0.22876667376141774, 'drop_l2': 0.10289910583836478, 'drop_l3': 0.3407393446642868, 'drop_l4': 0.6524882681282885, 'drop_l5': 0.47133828608582196, 'drop_l6': 0.12361226312038784, 'drop_l7': 0.4643670495351548, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:50:22,518] Trial 214 finished with value: 0.4195936322212219 and parameters: {'drop_l1': 0.23974349129365333, 'drop_l2': 0.14165160114799627, 'drop_l3': 0.24921468130602326, 'drop_l4': 0.6772134764184153, 'drop_l5': 0.47079111152696657, 'drop_l6': 0.12260397011261093, 'drop_l7': 0.46237922429766226, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:50:22,528] Trial 213 finished with value: 0.4586141109466553 and parameters: {'drop_l1': 0.37389219027391246, 'drop_l2': 0.13926202498764628, 'drop_l3': 0.7730663352616691, 'drop_l4': 0.7571890860476872, 'drop_l5': 0.4711946696304495, 'drop_l6': 0.1162718854402374, 'drop_l7': 0.7013106877094641, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:50:30,203] Trial 216 finished with value: 0.4465567469596863 and parameters: {'drop_l1': 0.2511725478727634, 'drop_l2': 0.10235163340474562, 'drop_l3': 0.8113984724416015, 'drop_l4': 0.6592058403372095, 'drop_l5': 0.4645956064150391, 'drop_l6': 0.11650371695681493, 'drop_l7': 0.6554252479989405, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:02,044] Trial 217 finished with value: 0.4492190480232239 and parameters: {'drop_l1': 0.24426035827822268, 'drop_l2': 0.1316550736087475, 'drop_l3': 0.10917888361518123, 'drop_l4': 0.7573715771321611, 'drop_l5': 0.43475826263462974, 'drop_l6': 0.11814939367695218, 'drop_l7': 0.6426245955073506, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:03,863] Trial 218 finished with value: 0.4663485884666443 and parameters: {'drop_l1': 0.28584723221645414, 'drop_l2': 0.10008910191116115, 'drop_l3': 0.1511436006435037, 'drop_l4': 0.6764478826803653, 'drop_l5': 0.43187485629695327, 'drop_l6': 0.13138746455899847, 'drop_l7': 0.4686600250701568, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:03,963] Trial 219 finished with value: 0.442472368478775 and parameters: {'drop_l1': 0.24643804694157087, 'drop_l2': 0.10067306148003666, 'drop_l3': 0.18162502581116047, 'drop_l4': 0.6641517239758735, 'drop_l5': 0.44118443646526406, 'drop_l6': 0.13490860614603414, 'drop_l7': 0.4662231512264634, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:06,981] Trial 220 finished with value: 0.4136173129081726 and parameters: {'drop_l1': 0.23827765277975402, 'drop_l2': 0.11666858469070128, 'drop_l3': 0.13024018203069943, 'drop_l4': 0.6803425287620587, 'drop_l5': 0.44514190127236475, 'drop_l6': 0.14809978943552352, 'drop_l7': 0.4717660112176662, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:43,041] Trial 221 finished with value: 0.4441758990287781 and parameters: {'drop_l1': 0.28754768410161596, 'drop_l2': 0.11260580889426403, 'drop_l3': 0.2257081637796644, 'drop_l4': 0.6406110892466116, 'drop_l5': 0.44266751208713717, 'drop_l6': 0.1462198944064588, 'drop_l7': 0.4704425003218654, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:44,266] Trial 223 finished with value: 0.4447537064552307 and parameters: {'drop_l1': 0.3212760106261749, 'drop_l2': 0.11910152730946195, 'drop_l3': 0.2854015331880839, 'drop_l4': 0.8408814974301725, 'drop_l5': 0.48414149102961046, 'drop_l6': 0.2071505165150956, 'drop_l7': 0.5546087980801391, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:44,752] Trial 222 finished with value: 0.45121970772743225 and parameters: {'drop_l1': 0.20346769651541913, 'drop_l2': 0.11633980375732601, 'drop_l3': 0.14918940465909797, 'drop_l4': 0.6460114932970646, 'drop_l5': 0.4440586413874685, 'drop_l6': 0.1483210185381522, 'drop_l7': 0.44293280752741887, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:51:45,340] Trial 224 finished with value: 0.4335218071937561 and parameters: {'drop_l1': 0.27076770964060654, 'drop_l2': 0.11687027835211879, 'drop_l3': 0.24756059530778773, 'drop_l4': 0.6404780798022512, 'drop_l5': 0.45423468505223097, 'drop_l6': 0.1457522919560077, 'drop_l7': 0.43912742971649066, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:52:23,929] Trial 225 finished with value: 0.4456603527069092 and parameters: {'drop_l1': 0.2694338176780533, 'drop_l2': 0.15137606892926822, 'drop_l3': 0.11720668973300753, 'drop_l4': 0.7164907099935669, 'drop_l5': 0.4897561640172866, 'drop_l6': 0.18054085502625036, 'drop_l7': 0.45175189317868064, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:52:25,013] Trial 228 finished with value: 0.4386076331138611 and parameters: {'drop_l1': 0.2174233745478521, 'drop_l2': 0.14685744513977256, 'drop_l3': 0.21275493491067468, 'drop_l4': 0.6778915549632325, 'drop_l5': 0.4716181728473661, 'drop_l6': 0.1027208109507043, 'drop_l7': 0.39728803580493083, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:52:26,123] Trial 227 finished with value: 0.42419758439064026 and parameters: {'drop_l1': 0.22314443496143171, 'drop_l2': 0.14245584645376663, 'drop_l3': 0.10061556780322271, 'drop_l4': 0.6796007140844909, 'drop_l5': 0.4919860401015269, 'drop_l6': 0.173923433875701, 'drop_l7': 0.5339023152393811, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:52:26,286] Trial 226 finished with value: 0.4393671154975891 and parameters: {'drop_l1': 0.26089784209363875, 'drop_l2': 0.1501580571723189, 'drop_l3': 0.12196405713114689, 'drop_l4': 0.6816087276809384, 'drop_l5': 0.4890403793385855, 'drop_l6': 0.1457784762365997, 'drop_l7': 0.5367474061955828, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:06,256] Trial 229 finished with value: 0.4400596618652344 and parameters: {'drop_l1': 0.18887991226021814, 'drop_l2': 0.13648021220980716, 'drop_l3': 0.2149557464845648, 'drop_l4': 0.5963175498929241, 'drop_l5': 0.4579917514480228, 'drop_l6': 0.10068825592499228, 'drop_l7': 0.536841254391821, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:07,008] Trial 230 finished with value: 0.4509000778198242 and parameters: {'drop_l1': 0.26319699200602026, 'drop_l2': 0.13309302897663922, 'drop_l3': 0.20241166525491808, 'drop_l4': 0.5973944541590006, 'drop_l5': 0.45929114290716855, 'drop_l6': 0.233574163498091, 'drop_l7': 0.5386799425080953, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:09,304] Trial 232 finished with value: 0.6382746696472168 and parameters: {'drop_l1': 0.31628012021383134, 'drop_l2': 0.13430608058594534, 'drop_l3': 0.12962095257171966, 'drop_l4': 0.5944832242451615, 'drop_l5': 0.4585990830678713, 'drop_l6': 0.22823983376101697, 'drop_l7': 0.5831234518330312, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:09,839] Trial 231 finished with value: 0.4461129307746887 and parameters: {'drop_l1': 0.17673565847346392, 'drop_l2': 0.1338268221621873, 'drop_l3': 0.1799987465118019, 'drop_l4': 0.679932002201482, 'drop_l5': 0.3924333460011471, 'drop_l6': 0.22352647362358546, 'drop_l7': 0.5263409554723953, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:54,140] Trial 234 finished with value: 0.41242092847824097 and parameters: {'drop_l1': 0.23245936099156964, 'drop_l2': 0.12030020208086055, 'drop_l3': 0.2795991639618754, 'drop_l4': 0.6633122168398135, 'drop_l5': 0.42442345459123015, 'drop_l6': 0.17913400665360346, 'drop_l7': 0.5799668565868652, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:54,347] Trial 233 finished with value: 0.6218338012695312 and parameters: {'drop_l1': 0.2333379573717364, 'drop_l2': 0.13466284879749377, 'drop_l3': 0.1408226740312329, 'drop_l4': 0.47131559340583207, 'drop_l5': 0.42617613833967094, 'drop_l6': 0.23281232000506602, 'drop_l7': 0.5225846411080461, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:57,347] Trial 236 finished with value: 0.4202814996242523 and parameters: {'drop_l1': 0.22545866313264787, 'drop_l2': 0.11617083382362527, 'drop_l3': 0.31613093796736275, 'drop_l4': 0.6607584150119052, 'drop_l5': 0.4765943064308372, 'drop_l6': 0.1756099575246406, 'drop_l7': 0.43239495444531756, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:53:57,610] Trial 235 finished with value: 0.4127330183982849 and parameters: {'drop_l1': 0.24290170266822128, 'drop_l2': 0.15859278731030652, 'drop_l3': 0.10539500376982902, 'drop_l4': 0.664066102774158, 'drop_l5': 0.4193669810759684, 'drop_l6': 0.1759567245555414, 'drop_l7': 0.4422139055777528, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:54:53,024] Trial 237 finished with value: 0.4303862452507019 and parameters: {'drop_l1': 0.21549304903998082, 'drop_l2': 0.15780636780439258, 'drop_l3': 0.29692496806735913, 'drop_l4': 0.6580826444658345, 'drop_l5': 0.4091124144408035, 'drop_l6': 0.20152417552531637, 'drop_l7': 0.5586430561193398, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:54:53,653] Trial 238 finished with value: 0.4362914562225342 and parameters: {'drop_l1': 0.22335989290995067, 'drop_l2': 0.1584244332010659, 'drop_l3': 0.10279932090577641, 'drop_l4': 0.6702704116280133, 'drop_l5': 0.41722140813153796, 'drop_l6': 0.17871364144251473, 'drop_l7': 0.5582317525854344, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:54:56,414] Trial 239 finished with value: 0.44768303632736206 and parameters: {'drop_l1': 0.20815374088323935, 'drop_l2': 0.10098258021286272, 'drop_l3': 0.281093572406969, 'drop_l4': 0.6516285566027498, 'drop_l5': 0.41464724864675473, 'drop_l6': 0.20069093589874507, 'drop_l7': 0.43296931184622545, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:54:56,416] Trial 240 finished with value: 0.4302077293395996 and parameters: {'drop_l1': 0.21334302689289614, 'drop_l2': 0.1224685427219242, 'drop_l3': 0.27154540365842655, 'drop_l4': 0.6580878987992572, 'drop_l5': 0.41838783191538303, 'drop_l6': 0.1998995144129987, 'drop_l7': 0.42717598093260023, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:55:48,640] Trial 241 finished with value: 0.41431769728660583 and parameters: {'drop_l1': 0.21584606316023824, 'drop_l2': 0.11307550501680826, 'drop_l3': 0.2822229538710051, 'drop_l4': 0.6609051397475553, 'drop_l5': 0.41418484659696625, 'drop_l6': 0.20091714281684608, 'drop_l7': 0.565339122818652, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:55:51,809] Trial 242 finished with value: 0.43249988555908203 and parameters: {'drop_l1': 0.20844798374972245, 'drop_l2': 0.10085491174999249, 'drop_l3': 0.25725897879437415, 'drop_l4': 0.6575144600543198, 'drop_l5': 0.4293482729176015, 'drop_l6': 0.20927673716239512, 'drop_l7': 0.43066444068466775, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:55:52,564] Trial 243 finished with value: 0.42560186982154846 and parameters: {'drop_l1': 0.19619897634861186, 'drop_l2': 0.12149067859972783, 'drop_l3': 0.2690921669402653, 'drop_l4': 0.6584162734678194, 'drop_l5': 0.42570148347384196, 'drop_l6': 0.20183753362562265, 'drop_l7': 0.45294986797292136, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:55:53,246] Trial 244 finished with value: 0.4560964107513428 and parameters: {'drop_l1': 0.19703160379294654, 'drop_l2': 0.11987351391422918, 'drop_l3': 0.30914826176600285, 'drop_l4': 0.6586340051586941, 'drop_l5': 0.42233991710218716, 'drop_l6': 0.1980340091254585, 'drop_l7': 0.580268669236598, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:56:41,875] Trial 245 finished with value: 0.42185431718826294 and parameters: {'drop_l1': 0.1951962469744827, 'drop_l2': 0.11461621153730187, 'drop_l3': 0.2952843668434874, 'drop_l4': 0.6622008713268456, 'drop_l5': 0.40500724680081795, 'drop_l6': 0.2003314819874901, 'drop_l7': 0.36688244761324, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:56:49,490] Trial 246 finished with value: 0.4590156674385071 and parameters: {'drop_l1': 0.195257992462059, 'drop_l2': 0.11171197008615377, 'drop_l3': 0.3119395568696672, 'drop_l4': 0.6583263551617137, 'drop_l5': 0.4085673389921054, 'drop_l6': 0.19088044653949301, 'drop_l7': 0.577240417124664, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:56:49,552] Trial 247 finished with value: 0.41976413130760193 and parameters: {'drop_l1': 0.2309688413100061, 'drop_l2': 0.12131848439673754, 'drop_l3': 0.311693539184137, 'drop_l4': 0.6692357006913919, 'drop_l5': 0.39120857258557806, 'drop_l6': 0.19440811239807618, 'drop_l7': 0.4216632462759424, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:56:49,565] Trial 248 finished with value: 0.4500060975551605 and parameters: {'drop_l1': 0.23164377281848542, 'drop_l2': 0.11860349154292887, 'drop_l3': 0.2835009946585887, 'drop_l4': 0.6662738524929809, 'drop_l5': 0.4096064567686292, 'drop_l6': 0.18762871235159295, 'drop_l7': 0.40602509668022263, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:57:33,028] Trial 249 finished with value: 0.4235258996486664 and parameters: {'drop_l1': 0.17887904016826675, 'drop_l2': 0.1158390079122867, 'drop_l3': 0.3098554624655501, 'drop_l4': 0.6631480883512306, 'drop_l5': 0.40534337292820843, 'drop_l6': 0.19378362318355713, 'drop_l7': 0.4174683814004312, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:57:44,932] Trial 252 finished with value: 0.4268549084663391 and parameters: {'drop_l1': 0.23886887736697213, 'drop_l2': 0.11844582655265751, 'drop_l3': 0.26590025502026554, 'drop_l4': 0.6218042548400936, 'drop_l5': 0.39564820799638983, 'drop_l6': 0.20593978210189925, 'drop_l7': 0.41515126463292007, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:57:45,311] Trial 250 finished with value: 0.4527181088924408 and parameters: {'drop_l1': 0.23712420296628428, 'drop_l2': 0.12167605923140751, 'drop_l3': 0.309325780596813, 'drop_l4': 0.625666286522778, 'drop_l5': 0.39022634822371505, 'drop_l6': 0.20459815856379715, 'drop_l7': 0.4086267136562152, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:57:45,898] Trial 251 finished with value: 0.4437492787837982 and parameters: {'drop_l1': 0.16921968967322312, 'drop_l2': 0.12297194999739652, 'drop_l3': 0.30509883742593535, 'drop_l4': 0.6200298979425619, 'drop_l5': 0.39648638421799326, 'drop_l6': 0.21307592551752164, 'drop_l7': 0.36261339099439077, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:58:21,295] Trial 253 finished with value: 0.40770381689071655 and parameters: {'drop_l1': 0.17048697366931115, 'drop_l2': 0.12387062243757532, 'drop_l3': 0.2652831396918145, 'drop_l4': 0.6300648334037086, 'drop_l5': 0.39469890782429307, 'drop_l6': 0.20388077432982848, 'drop_l7': 0.3424373457596773, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:58:40,979] Trial 255 finished with value: 0.44522613286972046 and parameters: {'drop_l1': 0.18512415434917362, 'drop_l2': 0.11004665288200834, 'drop_l3': 0.2686798440743917, 'drop_l4': 0.6348502178383221, 'drop_l5': 0.434440676807143, 'drop_l6': 0.1825281524412149, 'drop_l7': 0.3820330497062186, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:58:42,169] Trial 256 finished with value: 0.5211988687515259 and parameters: {'drop_l1': 0.18382820571010794, 'drop_l2': 0.10109665497840331, 'drop_l3': 0.26363472477610356, 'drop_l4': 0.6373986569438427, 'drop_l5': 0.43215474907786716, 'drop_l6': 0.17815069721333246, 'drop_l7': 0.406931777916937, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:58:42,231] Trial 254 finished with value: 0.42979687452316284 and parameters: {'drop_l1': 0.1676248812449747, 'drop_l2': 0.1174496609859463, 'drop_l3': 0.2704811404390576, 'drop_l4': 0.6243031307252559, 'drop_l5': 0.3939423450171589, 'drop_l6': 0.21416707367900517, 'drop_l7': 0.3683818132044366, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:59:10,593] Trial 257 finished with value: 0.519233763217926 and parameters: {'drop_l1': 0.1299981460741608, 'drop_l2': 0.10338495371096483, 'drop_l3': 0.26520468159697913, 'drop_l4': 0.6889369858257492, 'drop_l5': 0.43833605644422824, 'drop_l6': 0.17971119912412595, 'drop_l7': 0.3177099182593122, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:59:35,597] Trial 258 finished with value: 0.4161674380302429 and parameters: {'drop_l1': 0.141981025444482, 'drop_l2': 0.10816329961511799, 'drop_l3': 0.33355968653063656, 'drop_l4': 0.6893161428307545, 'drop_l5': 0.3861089356889502, 'drop_l6': 0.17607117209782683, 'drop_l7': 0.3105785588657435, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:59:38,715] Trial 259 finished with value: 0.44220954179763794 and parameters: {'drop_l1': 0.15384326194589454, 'drop_l2': 0.10409387484652839, 'drop_l3': 0.3286415743332077, 'drop_l4': 0.6960520907975631, 'drop_l5': 0.3873577530621624, 'drop_l6': 0.18125401212740763, 'drop_l7': 0.41820246086377055, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:59:38,747] Trial 260 finished with value: 0.4599929451942444 and parameters: {'drop_l1': 0.13364094016323, 'drop_l2': 0.13289069588363778, 'drop_l3': 0.3191293727609591, 'drop_l4': 0.6887835039419707, 'drop_l5': 0.396371060765818, 'drop_l6': 0.17831857702412723, 'drop_l7': 0.3332079258850998, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 17:59:59,690] Trial 261 finished with value: 0.43653589487075806 and parameters: {'drop_l1': 0.24072452633222385, 'drop_l2': 0.10036078341934365, 'drop_l3': 0.24318968870268473, 'drop_l4': 0.6867671122309583, 'drop_l5': 0.39090831790916536, 'drop_l6': 0.17319681893066813, 'drop_l7': 0.38057717108641004, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:00:33,725] Trial 262 finished with value: 0.41958245635032654 and parameters: {'drop_l1': 0.128293210645643, 'drop_l2': 0.13473269512281594, 'drop_l3': 0.3347482997034162, 'drop_l4': 0.6855139152241113, 'drop_l5': 0.3849881173273638, 'drop_l6': 0.19450922658555236, 'drop_l7': 0.35020486020402436, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:00:37,148] Trial 264 finished with value: 0.4557202458381653 and parameters: {'drop_l1': 0.23995848078624255, 'drop_l2': 0.13868037182489587, 'drop_l3': 0.37107206543602517, 'drop_l4': 0.6688749308373223, 'drop_l5': 0.4001911148306618, 'drop_l6': 0.1971484625502789, 'drop_l7': 0.3388088866560909, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:00:37,882] Trial 263 finished with value: 0.4710538983345032 and parameters: {'drop_l1': 0.13846810889120217, 'drop_l2': 0.1344072963210472, 'drop_l3': 0.2476800189614869, 'drop_l4': 0.6118050352413821, 'drop_l5': 0.37875081617063966, 'drop_l6': 0.19442204702353685, 'drop_l7': 0.2947423044924606, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:00:52,756] Trial 265 finished with value: 0.4339275360107422 and parameters: {'drop_l1': 0.21645063575046414, 'drop_l2': 0.1384985772874793, 'drop_l3': 0.33164684360125896, 'drop_l4': 0.6704578710262968, 'drop_l5': 0.405039112898285, 'drop_l6': 0.20543588638256718, 'drop_l7': 0.4638348307668606, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:01:30,414] Trial 266 finished with value: 0.48001623153686523 and parameters: {'drop_l1': 0.10075154336236981, 'drop_l2': 0.13173178289699558, 'drop_l3': 0.3670937918082554, 'drop_l4': 0.6718900053061672, 'drop_l5': 0.37903366478010764, 'drop_l6': 0.21827806664537902, 'drop_l7': 0.2945247612307831, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:01:35,606] Trial 267 finished with value: 0.47530198097229004 and parameters: {'drop_l1': 0.19780294714969796, 'drop_l2': 0.12743141958415868, 'drop_l3': 0.34634515203395455, 'drop_l4': 0.6429361977881421, 'drop_l5': 0.380538207782381, 'drop_l6': 0.22049458519449733, 'drop_l7': 0.30308053575968485, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:01:36,237] Trial 268 finished with value: 0.4339037835597992 and parameters: {'drop_l1': 0.16159576068057124, 'drop_l2': 0.12504553627668466, 'drop_l3': 0.35195155975720266, 'drop_l4': 0.6434583352177298, 'drop_l5': 0.37961957664433266, 'drop_l6': 0.21932786246874034, 'drop_l7': 0.30864907523723156, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:01:45,943] Trial 269 finished with value: 0.4980584979057312 and parameters: {'drop_l1': 0.19591200352481422, 'drop_l2': 0.12386192653614528, 'drop_l3': 0.3446607401317049, 'drop_l4': 0.6421727051429733, 'drop_l5': 0.38280056680130603, 'drop_l6': 0.22070736622958897, 'drop_l7': 0.3547890175271505, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:02:26,395] Trial 270 finished with value: 0.5054872035980225 and parameters: {'drop_l1': 0.16608130919682418, 'drop_l2': 0.11681379638943046, 'drop_l3': 0.35267488382965073, 'drop_l4': 0.6433850859185033, 'drop_l5': 0.42067647949835385, 'drop_l6': 0.21984434113258222, 'drop_l7': 0.3478858692966541, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:02:32,995] Trial 271 finished with value: 0.44611573219299316 and parameters: {'drop_l1': 0.11969689738053706, 'drop_l2': 0.10020051647517508, 'drop_l3': 0.29231080721150426, 'drop_l4': 0.6393188051453681, 'drop_l5': 0.35549593345495556, 'drop_l6': 0.16465364670807428, 'drop_l7': 0.33242802910573005, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:02:33,236] Trial 272 finished with value: 0.4632893204689026 and parameters: {'drop_l1': 0.12375175546032312, 'drop_l2': 0.6478795187568115, 'drop_l3': 0.4014676984219183, 'drop_l4': 0.7025072405297637, 'drop_l5': 0.352043924094976, 'drop_l6': 0.1683283261549984, 'drop_l7': 0.3546791065518796, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:02:38,545] Trial 273 finished with value: 0.4222164750099182 and parameters: {'drop_l1': 0.12000669105378574, 'drop_l2': 0.10485241750850177, 'drop_l3': 0.29976632104228024, 'drop_l4': 0.7029558767352988, 'drop_l5': 0.41639365865065625, 'drop_l6': 0.1728868348036568, 'drop_l7': 0.38613397124059073, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:03:14,886] Trial 274 finished with value: 0.42502260208129883 and parameters: {'drop_l1': 0.22391195650744802, 'drop_l2': 0.10133056001675858, 'drop_l3': 0.29407150063778525, 'drop_l4': 0.7010275514820569, 'drop_l5': 0.35000196824235874, 'drop_l6': 0.16956559683708877, 'drop_l7': 0.4476512961930242, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:03:24,605] Trial 275 finished with value: 0.432913213968277 and parameters: {'drop_l1': 0.22165446172358116, 'drop_l2': 0.10171495359845703, 'drop_l3': 0.2966145735703429, 'drop_l4': 0.6994384442882863, 'drop_l5': 0.4101212339811988, 'drop_l6': 0.18968766035485277, 'drop_l7': 0.37720727273840765, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:03:24,612] Trial 276 finished with value: 0.430044561624527 and parameters: {'drop_l1': 0.22433409691244538, 'drop_l2': 0.10002450227919277, 'drop_l3': 0.297706176063074, 'drop_l4': 0.6792684735776092, 'drop_l5': 0.4457858191600128, 'drop_l6': 0.1912158215841929, 'drop_l7': 0.3802248778035448, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:03:26,533] Trial 277 finished with value: 0.4414688050746918 and parameters: {'drop_l1': 0.10798163985333364, 'drop_l2': 0.14587038464799615, 'drop_l3': 0.2963494258357389, 'drop_l4': 0.6796000777317411, 'drop_l5': 0.421443460630068, 'drop_l6': 0.19822989599868318, 'drop_l7': 0.3897519594492091, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:03:53,560] Trial 278 finished with value: 0.44765812158584595 and parameters: {'drop_l1': 0.2260799117963811, 'drop_l2': 0.10646002635808921, 'drop_l3': 0.2949819110664462, 'drop_l4': 0.6997680920507724, 'drop_l5': 0.4057882067817813, 'drop_l6': 0.19326819843475532, 'drop_l7': 0.3761624050949665, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:04:15,289] Trial 279 finished with value: 0.4208087921142578 and parameters: {'drop_l1': 0.145405380667218, 'drop_l2': 0.3295806007878861, 'drop_l3': 0.2783085998482873, 'drop_l4': 0.7082024888595668, 'drop_l5': 0.4234419814892358, 'drop_l6': 0.16929111981266742, 'drop_l7': 0.3936551478399942, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:04:16,446] Trial 280 finished with value: 0.42562156915664673 and parameters: {'drop_l1': 0.23965529914638056, 'drop_l2': 0.14608694449419718, 'drop_l3': 0.31881921345365594, 'drop_l4': 0.7102264762354522, 'drop_l5': 0.42302688896946367, 'drop_l6': 0.1659594649303337, 'drop_l7': 0.392386894049975, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:04:16,493] Trial 281 finished with value: 0.4477016031742096 and parameters: {'drop_l1': 0.14642663731575842, 'drop_l2': 0.11037584680428751, 'drop_l3': 0.3159571050628646, 'drop_l4': 0.7042167070994103, 'drop_l5': 0.769585769533367, 'drop_l6': 0.16811316807679472, 'drop_l7': 0.44634889075828016, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:04:34,096] Trial 282 finished with value: 0.5037192106246948 and parameters: {'drop_l1': 0.1837351743331911, 'drop_l2': 0.14446872647497547, 'drop_l3': 0.31481306215307026, 'drop_l4': 0.665191711527043, 'drop_l5': 0.7307156452594994, 'drop_l6': 0.17166274476570995, 'drop_l7': 0.44530624016293424, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:04,637] Trial 283 finished with value: 0.4409428536891937 and parameters: {'drop_l1': 0.17957861399007946, 'drop_l2': 0.14192884054626786, 'drop_l3': 0.3244659699731714, 'drop_l4': 0.7113942090645476, 'drop_l5': 0.4281496029223019, 'drop_l6': 0.17182114103884794, 'drop_l7': 0.44929979121448016, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:05,391] Trial 284 finished with value: 0.45202571153640747 and parameters: {'drop_l1': 0.1450642695368836, 'drop_l2': 0.3812117163687654, 'drop_l3': 0.28253524198713303, 'drop_l4': 0.6584423944773445, 'drop_l5': 0.44670792559275446, 'drop_l6': 0.17230646217168347, 'drop_l7': 0.3231490057187143, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:07,574] Trial 285 finished with value: 0.41835489869117737 and parameters: {'drop_l1': 0.1863275478879436, 'drop_l2': 0.14594641751321819, 'drop_l3': 0.27538616940100874, 'drop_l4': 0.6686166999870817, 'drop_l5': 0.44171050996815325, 'drop_l6': 0.18097359585697476, 'drop_l7': 0.44406053069888884, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:15,275] Trial 286 finished with value: 0.4327862858772278 and parameters: {'drop_l1': 0.1502336363114537, 'drop_l2': 0.49456397265945784, 'drop_l3': 0.3339207042925779, 'drop_l4': 0.6839367209952724, 'drop_l5': 0.4448846322185874, 'drop_l6': 0.14861332441065878, 'drop_l7': 0.42813302627062655, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:53,860] Trial 287 finished with value: 0.5156640410423279 and parameters: {'drop_l1': 0.14957007028115035, 'drop_l2': 0.10053083621175674, 'drop_l3': 0.28518638390954787, 'drop_l4': 0.6899557551833673, 'drop_l5': 0.44824131211086926, 'drop_l6': 0.15380109404179856, 'drop_l7': 0.320512086255204, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:55,367] Trial 288 finished with value: 0.5259628295898438 and parameters: {'drop_l1': 0.2527927028621316, 'drop_l2': 0.4219829274504262, 'drop_l3': 0.27485560183196156, 'drop_l4': 0.6933298665715479, 'drop_l5': 0.3386460125077851, 'drop_l6': 0.15422489825991134, 'drop_l7': 0.42430213592600674, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:05:59,266] Trial 289 finished with value: 0.4291302561759949 and parameters: {'drop_l1': 0.25536976492959695, 'drop_l2': 0.15088873681547027, 'drop_l3': 0.2754765578515113, 'drop_l4': 0.6860210569675851, 'drop_l5': 0.3380663490296152, 'drop_l6': 0.14778098009854168, 'drop_l7': 0.42931412387846807, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:06:01,685] Trial 290 finished with value: 0.49420905113220215 and parameters: {'drop_l1': 0.25181456209615743, 'drop_l2': 0.15232628476399868, 'drop_l3': 0.2871683167401832, 'drop_l4': 0.6898041275015162, 'drop_l5': 0.3454291096184742, 'drop_l6': 0.15217027120774304, 'drop_l7': 0.4033378453812662, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:06:42,137] Trial 291 finished with value: 0.5077211856842041 and parameters: {'drop_l1': 0.2506072605730933, 'drop_l2': 0.1515922066980345, 'drop_l3': 0.10236576145791439, 'drop_l4': 0.6750466160575799, 'drop_l5': 0.34013804502951295, 'drop_l6': 0.1780551796455643, 'drop_l7': 0.41615328610948216, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:06:44,366] Trial 292 finished with value: 0.4568381905555725 and parameters: {'drop_l1': 0.17082986171380202, 'drop_l2': 0.1573040272892997, 'drop_l3': 0.2290408788588328, 'drop_l4': 0.6724316978672354, 'drop_l5': 0.3915438811384407, 'drop_l6': 0.17988014036749445, 'drop_l7': 0.36092285847415284, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:06:48,747] Trial 293 finished with value: 0.4856339395046234 and parameters: {'drop_l1': 0.2066082834005247, 'drop_l2': 0.3396043061088317, 'drop_l3': 0.30063559593523653, 'drop_l4': 0.6697942301985778, 'drop_l5': 0.35792474464373314, 'drop_l6': 0.18702326357423565, 'drop_l7': 0.41254443278848696, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:06:49,166] Trial 294 finished with value: 0.4968357980251312 and parameters: {'drop_l1': 0.20849920764645608, 'drop_l2': 0.13431890194186258, 'drop_l3': 0.24817293396269188, 'drop_l4': 0.6695303701618702, 'drop_l5': 0.36434513711726346, 'drop_l6': 0.1868503716783766, 'drop_l7': 0.46421961987341887, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:07:31,081] Trial 295 finished with value: 0.4690244197845459 and parameters: {'drop_l1': 0.17455997564673487, 'drop_l2': 0.28976919271666435, 'drop_l3': 0.2412529384398172, 'drop_l4': 0.6622336464303595, 'drop_l5': 0.3652239175774761, 'drop_l6': 0.18622430021120676, 'drop_l7': 0.4761606696000923, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:07:35,977] Trial 296 finished with value: 0.49342218041419983 and parameters: {'drop_l1': 0.11286107447242684, 'drop_l2': 0.2873353099318275, 'drop_l3': 0.24468872719650941, 'drop_l4': 0.6564043110910699, 'drop_l5': 0.40701722831131626, 'drop_l6': 0.13301310283566184, 'drop_l7': 0.3952143228114319, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:07:39,304] Trial 298 finished with value: 0.4185967445373535 and parameters: {'drop_l1': 0.11787224918016473, 'drop_l2': 0.29508458248519376, 'drop_l3': 0.3051454914834112, 'drop_l4': 0.716177838334627, 'drop_l5': 0.40956848958735104, 'drop_l6': 0.13363458644559126, 'drop_l7': 0.4766864171716912, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:07:39,306] Trial 297 finished with value: 0.4591708183288574 and parameters: {'drop_l1': 0.1038514029173606, 'drop_l2': 0.13185315519066687, 'drop_l3': 0.25109793034190786, 'drop_l4': 0.7139194431332515, 'drop_l5': 0.8801114339457174, 'drop_l6': 0.13183510142507404, 'drop_l7': 0.4578386121151658, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:08:14,922] Trial 299 finished with value: 0.4337947368621826 and parameters: {'drop_l1': 0.1151445242044429, 'drop_l2': 0.11421771028980476, 'drop_l3': 0.3274191958157328, 'drop_l4': 0.7122000112696388, 'drop_l5': 0.4059386349290937, 'drop_l6': 0.12459291662135892, 'drop_l7': 0.395904557144782, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:08:26,224] Trial 300 finished with value: 0.8591998815536499 and parameters: {'drop_l1': 0.22085829749057997, 'drop_l2': 0.10109703909189345, 'drop_l3': 0.3036235311835234, 'drop_l4': 0.7155227750699636, 'drop_l5': 0.4356834846717024, 'drop_l6': 0.13659245297511538, 'drop_l7': 0.44583210289460273, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:08:29,029] Trial 302 finished with value: 0.4467410147190094 and parameters: {'drop_l1': 0.14013598448641118, 'drop_l2': 0.33070167739324946, 'drop_l3': 0.30877688275929993, 'drop_l4': 0.7154924991223671, 'drop_l5': 0.4132312828392257, 'drop_l6': 0.11554310682545188, 'drop_l7': 0.3618903656771094, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:08:29,551] Trial 301 finished with value: 0.8959723711013794 and parameters: {'drop_l1': 0.12429842205119243, 'drop_l2': 0.351659013730471, 'drop_l3': 0.3295033135737202, 'drop_l4': 0.7097452387497486, 'drop_l5': 0.42014048502289314, 'drop_l6': 0.1156117408922314, 'drop_l7': 0.4378866881306782, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:08:52,854] Trial 303 finished with value: 0.9160062074661255 and parameters: {'drop_l1': 0.13767277766497882, 'drop_l2': 0.2647411095768274, 'drop_l3': 0.3015166646802488, 'drop_l4': 0.7311932094557363, 'drop_l5': 0.43327059099286086, 'drop_l6': 0.1025586382450043, 'drop_l7': 0.44313421255735436, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:09:14,425] Trial 304 finished with value: 0.4138201177120209 and parameters: {'drop_l1': 0.13587124962505948, 'drop_l2': 0.3517745420391603, 'drop_l3': 0.3219255234535955, 'drop_l4': 0.7274939137277149, 'drop_l5': 0.42043180367149513, 'drop_l6': 0.11604311691005435, 'drop_l7': 0.43349160108809737, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 157 with value: 0.4069921374320984. [I 2021-03-10 18:09:17,089] Trial 305 finished with value: 0.40107059478759766 and parameters: {'drop_l1': 0.12663821571673492, 'drop_l2': 0.30239507806677624, 'drop_l3': 0.282909865243885, 'drop_l4': 0.6908456120963694, 'drop_l5': 0.42632508457851687, 'drop_l6': 0.16171739450681577, 'drop_l7': 0.4356194526560724, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:09:17,623] Trial 306 finished with value: 0.4486027657985687 and parameters: {'drop_l1': 0.18979373131042826, 'drop_l2': 0.2970558996881271, 'drop_l3': 0.28774561177415553, 'drop_l4': 0.6911655333667284, 'drop_l5': 0.3937847100401525, 'drop_l6': 0.16096603834246373, 'drop_l7': 0.4646924710954587, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:09:27,329] Trial 307 finished with value: 0.46848756074905396 and parameters: {'drop_l1': 0.16134615245560724, 'drop_l2': 0.10096996310005871, 'drop_l3': 0.3593998165844994, 'drop_l4': 0.6953752912723755, 'drop_l5': 0.4003548959870665, 'drop_l6': 0.16008841693981707, 'drop_l7': 0.33655188778275497, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:10:04,958] Trial 308 finished with value: 0.4751354157924652 and parameters: {'drop_l1': 0.15432308069556608, 'drop_l2': 0.3304419709763813, 'drop_l3': 0.31537548696385437, 'drop_l4': 0.7272458122719614, 'drop_l5': 0.6697688359446521, 'drop_l6': 0.10297747682566705, 'drop_l7': 0.3423068934156989, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:10:08,070] Trial 309 finished with value: 0.4262634217739105 and parameters: {'drop_l1': 0.16046469719260792, 'drop_l2': 0.31296087717930055, 'drop_l3': 0.3616452100529413, 'drop_l4': 0.726683906765159, 'drop_l5': 0.40098129718581366, 'drop_l6': 0.15642472383292638, 'drop_l7': 0.34198311238786583, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:10:08,386] Trial 310 finished with value: 0.41909679770469666 and parameters: {'drop_l1': 0.1291897891200474, 'drop_l2': 0.31612940677706036, 'drop_l3': 0.3252651954721122, 'drop_l4': 0.43722639424236154, 'drop_l5': 0.4204994951466031, 'drop_l6': 0.1406073769014392, 'drop_l7': 0.4237598379428825, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:10:16,573] Trial 311 finished with value: 0.44852596521377563 and parameters: {'drop_l1': 0.12300416884953523, 'drop_l2': 0.3573472612792625, 'drop_l3': 0.33527175697634426, 'drop_l4': 0.7274350517472907, 'drop_l5': 0.42567142211934206, 'drop_l6': 0.14246700184023817, 'drop_l7': 0.4245327781880187, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:10:59,660] Trial 312 finished with value: 0.4048430323600769 and parameters: {'drop_l1': 0.11640577517135846, 'drop_l2': 0.3619390137811864, 'drop_l3': 0.339614276549912, 'drop_l4': 0.6786303266602415, 'drop_l5': 0.42289730338859716, 'drop_l6': 0.14311700740664807, 'drop_l7': 0.42355165962314584, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:11:04,563] Trial 314 finished with value: 0.4326145648956299 and parameters: {'drop_l1': 0.1275618389334016, 'drop_l2': 0.31684362048164993, 'drop_l3': 0.32473317856702555, 'drop_l4': 0.46732678512709724, 'drop_l5': 0.42488756906711744, 'drop_l6': 0.14259495577529108, 'drop_l7': 0.4213773345500585, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:11:05,331] Trial 313 finished with value: 0.41799670457839966 and parameters: {'drop_l1': 0.12599545554566136, 'drop_l2': 0.3524687119907005, 'drop_l3': 0.33430163008512287, 'drop_l4': 0.6841340380262382, 'drop_l5': 0.4205331304001056, 'drop_l6': 0.1458127637332779, 'drop_l7': 0.42696346584548023, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:11:10,004] Trial 315 finished with value: 0.4915396273136139 and parameters: {'drop_l1': 0.10805756836888647, 'drop_l2': 0.3177188462314668, 'drop_l3': 0.31743871829336767, 'drop_l4': 0.4169584075891314, 'drop_l5': 0.4231430144094439, 'drop_l6': 0.3708231179083352, 'drop_l7': 0.4069601027018521, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:11:53,643] Trial 316 finished with value: 0.4884059429168701 and parameters: {'drop_l1': 0.10074317006367067, 'drop_l2': 0.308886249674505, 'drop_l3': 0.3809278003924654, 'drop_l4': 0.503454823668907, 'drop_l5': 0.44036513622912976, 'drop_l6': 0.39653640378904187, 'drop_l7': 0.40066750125653566, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:01,954] Trial 317 finished with value: 0.4249279499053955 and parameters: {'drop_l1': 0.108135432314654, 'drop_l2': 0.31912654693653486, 'drop_l3': 0.3453638258570181, 'drop_l4': 0.4191764651697524, 'drop_l5': 0.44747662752587997, 'drop_l6': 0.1278663549092338, 'drop_l7': 0.40274622974132296, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:02,086] Trial 318 finished with value: 0.41828662157058716 and parameters: {'drop_l1': 0.11259103788284355, 'drop_l2': 0.3645366277980682, 'drop_l3': 0.34251139042283346, 'drop_l4': 0.4101102074623043, 'drop_l5': 0.4445429110415782, 'drop_l6': 0.13217528975887857, 'drop_l7': 0.4334932842279984, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:03,972] Trial 319 finished with value: 0.42135006189346313 and parameters: {'drop_l1': 0.13242143771254103, 'drop_l2': 0.3738206723110232, 'drop_l3': 0.34118994277378345, 'drop_l4': 0.4366128558942254, 'drop_l5': 0.4476030090747478, 'drop_l6': 0.13085009374728404, 'drop_l7': 0.45875071330136113, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:45,627] Trial 320 finished with value: 0.4025123119354248 and parameters: {'drop_l1': 0.13508654993371136, 'drop_l2': 0.36527076132956504, 'drop_l3': 0.3409153699368952, 'drop_l4': 0.6756220650600312, 'drop_l5': 0.45010482210335034, 'drop_l6': 0.13200795821771716, 'drop_l7': 0.4357637783579014, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:56,810] Trial 321 finished with value: 0.43622952699661255 and parameters: {'drop_l1': 0.1373031030790877, 'drop_l2': 0.35011951103137035, 'drop_l3': 0.3471929768527419, 'drop_l4': 0.4245388078839128, 'drop_l5': 0.4612408822107582, 'drop_l6': 0.13763904604926513, 'drop_l7': 0.4624286347583605, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:58,450] Trial 322 finished with value: 0.43236875534057617 and parameters: {'drop_l1': 0.12552748644997974, 'drop_l2': 0.3715811669403874, 'drop_l3': 0.34120118105023434, 'drop_l4': 0.36661617459706525, 'drop_l5': 0.45723203621489367, 'drop_l6': 0.1359899627904137, 'drop_l7': 0.43205885935636335, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:12:59,819] Trial 323 finished with value: 0.42938360571861267 and parameters: {'drop_l1': 0.14135557999891576, 'drop_l2': 0.3586510277037083, 'drop_l3': 0.3527251765806018, 'drop_l4': 0.40014992669884736, 'drop_l5': 0.43294938347760825, 'drop_l6': 0.11239353515583887, 'drop_l7': 0.4293950272668036, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:13:33,143] Trial 324 finished with value: 0.489143043756485 and parameters: {'drop_l1': 0.10097885292403642, 'drop_l2': 0.3637076471042393, 'drop_l3': 0.37885514584828883, 'drop_l4': 0.4103514120529485, 'drop_l5': 0.4592058229348442, 'drop_l6': 0.11034220644329515, 'drop_l7': 0.4298121659634683, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:13:54,059] Trial 325 finished with value: 0.5468371510505676 and parameters: {'drop_l1': 0.11946300602992237, 'drop_l2': 0.4043253984047981, 'drop_l3': 0.3698612608130733, 'drop_l4': 0.6803211964973507, 'drop_l5': 0.4576361565337935, 'drop_l6': 0.10070488255451915, 'drop_l7': 0.4366750803304702, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:13:54,715] Trial 326 finished with value: 0.5268492698669434 and parameters: {'drop_l1': 0.10646237778902787, 'drop_l2': 0.3965100322477368, 'drop_l3': 0.3671819042355395, 'drop_l4': 0.671213061591377, 'drop_l5': 0.4371745233634227, 'drop_l6': 0.11613017862036118, 'drop_l7': 0.47344229870149873, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:13:57,281] Trial 327 finished with value: 0.4771636426448822 and parameters: {'drop_l1': 0.10389174934488352, 'drop_l2': 0.3954204470739356, 'drop_l3': 0.37588191025318124, 'drop_l4': 0.6812982674052763, 'drop_l5': 0.45636896971658647, 'drop_l6': 0.11167489516807025, 'drop_l7': 0.2790333280402605, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:14:21,835] Trial 328 finished with value: 0.4235662817955017 and parameters: {'drop_l1': 0.11814201548769697, 'drop_l2': 0.40516210111435275, 'drop_l3': 0.35235412493333934, 'drop_l4': 0.4462301524565243, 'drop_l5': 0.4371314325496646, 'drop_l6': 0.11799964845831742, 'drop_l7': 0.27145092481395083, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:14:51,735] Trial 330 finished with value: 0.514586329460144 and parameters: {'drop_l1': 0.13074149445587596, 'drop_l2': 0.34404266128445415, 'drop_l3': 0.32814104730291815, 'drop_l4': 0.38977492535839664, 'drop_l5': 0.4461262462444916, 'drop_l6': 0.14331986761538495, 'drop_l7': 0.4777792081785483, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:14:51,737] Trial 329 finished with value: 0.5043972730636597 and parameters: {'drop_l1': 0.10120670074572692, 'drop_l2': 0.3412399702315342, 'drop_l3': 0.3929877001626102, 'drop_l4': 0.6823200147227675, 'drop_l5': 0.4339834081633559, 'drop_l6': 0.12174758219946034, 'drop_l7': 0.2696732717532195, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:14:55,106] Trial 331 finished with value: 0.4396580755710602 and parameters: {'drop_l1': 0.13565111993403778, 'drop_l2': 0.2777026743503925, 'drop_l3': 0.32446748653819774, 'drop_l4': 0.4389379789434022, 'drop_l5': 0.4423720635705826, 'drop_l6': 0.14100599674759595, 'drop_l7': 0.45380810969920926, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:15:11,218] Trial 332 finished with value: 0.45964184403419495 and parameters: {'drop_l1': 0.1355575005778336, 'drop_l2': 0.33889251952209487, 'drop_l3': 0.3234935847796046, 'drop_l4': 0.39216755786803886, 'drop_l5': 0.38611877958546253, 'drop_l6': 0.1469298491776035, 'drop_l7': 0.4444136543515922, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:15:48,277] Trial 334 finished with value: 0.46922674775123596 and parameters: {'drop_l1': 0.15957756651349464, 'drop_l2': 0.27977532467169647, 'drop_l3': 0.33239682220893335, 'drop_l4': 0.44678321262051995, 'drop_l5': 0.4188699192748741, 'drop_l6': 0.13806201595856316, 'drop_l7': 0.45344931359206037, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:15:49,183] Trial 333 finished with value: 0.45826882123947144 and parameters: {'drop_l1': 0.15358900600474168, 'drop_l2': 0.37127001066743337, 'drop_l3': 0.3341815347726986, 'drop_l4': 0.4558852770166371, 'drop_l5': 0.3843189262299789, 'drop_l6': 0.14859623654721507, 'drop_l7': 0.45414335596404704, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:15:53,430] Trial 335 finished with value: 0.43840765953063965 and parameters: {'drop_l1': 0.1560612615020125, 'drop_l2': 0.30432924581579546, 'drop_l3': 0.33676273611514407, 'drop_l4': 0.45792595193381813, 'drop_l5': 0.38571144347408026, 'drop_l6': 0.14930468375934147, 'drop_l7': 0.4455083220297155, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:16:02,691] Trial 336 finished with value: 0.4265299439430237 and parameters: {'drop_l1': 0.15983078868458012, 'drop_l2': 0.36649400122611225, 'drop_l3': 0.41144323479865086, 'drop_l4': 0.6893330232218925, 'drop_l5': 0.41680273164510034, 'drop_l6': 0.10070532961987336, 'drop_l7': 0.48132341331179834, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:16:41,988] Trial 337 finished with value: 0.43407344818115234 and parameters: {'drop_l1': 0.27293952104364716, 'drop_l2': 0.38233593523545945, 'drop_l3': 0.3122948813979017, 'drop_l4': 0.5333008715592922, 'drop_l5': 0.3867067004436329, 'drop_l6': 0.1517144972676302, 'drop_l7': 0.47791405693176164, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:16:44,170] Trial 338 finished with value: 0.4472060799598694 and parameters: {'drop_l1': 0.1180602893560807, 'drop_l2': 0.36086752812840545, 'drop_l3': 0.2605507244874677, 'drop_l4': 0.5324162156383222, 'drop_l5': 0.6355737729957149, 'drop_l6': 0.1291645682693282, 'drop_l7': 0.48486702483144045, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:16:49,675] Trial 339 finished with value: 0.5064558982849121 and parameters: {'drop_l1': 0.2644316311827829, 'drop_l2': 0.34973155209992624, 'drop_l3': 0.31311931043144114, 'drop_l4': 0.6473487944164468, 'drop_l5': 0.4065254067049548, 'drop_l6': 0.10122927777332558, 'drop_l7': 0.4844066646905897, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:16:54,571] Trial 340 finished with value: 0.530145525932312 and parameters: {'drop_l1': 0.2771821531478648, 'drop_l2': 0.38790093648613205, 'drop_l3': 0.2603043756416352, 'drop_l4': 0.6503585385121299, 'drop_l5': 0.41151282600611666, 'drop_l6': 0.12410242614928994, 'drop_l7': 0.4158131796058398, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:17:37,045] Trial 341 finished with value: 0.47898977994918823 and parameters: {'drop_l1': 0.23383968928168034, 'drop_l2': 0.3503124348396153, 'drop_l3': 0.279390153142403, 'drop_l4': 0.5176979590228337, 'drop_l5': 0.4036411800570259, 'drop_l6': 0.12760099053153218, 'drop_l7': 0.41746712544737, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:17:42,125] Trial 342 finished with value: 0.5062810778617859 and parameters: {'drop_l1': 0.2822914287028237, 'drop_l2': 0.32205809384747236, 'drop_l3': 0.28463193105222534, 'drop_l4': 0.6481919655245324, 'drop_l5': 0.4078278567544345, 'drop_l6': 0.4919608323736745, 'drop_l7': 0.434254140745558, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:17:44,732] Trial 343 finished with value: 0.419999897480011 and parameters: {'drop_l1': 0.602408165863374, 'drop_l2': 0.31960773111237173, 'drop_l3': 0.28055415023499386, 'drop_l4': 0.6518893731690764, 'drop_l5': 0.4656614797486168, 'drop_l6': 0.1280745244332382, 'drop_l7': 0.4348378043291853, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:17:47,231] Trial 344 finished with value: 0.41758936643600464 and parameters: {'drop_l1': 0.20861810598868527, 'drop_l2': 0.33350325063587855, 'drop_l3': 0.2838166984573664, 'drop_l4': 0.6314393961321815, 'drop_l5': 0.46635682039162, 'drop_l6': 0.16347884985543282, 'drop_l7': 0.43151378167556775, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:18:31,043] Trial 345 finished with value: 0.5285239815711975 and parameters: {'drop_l1': 0.12196920887450112, 'drop_l2': 0.32842963857745433, 'drop_l3': 0.2849576747795144, 'drop_l4': 0.6965775232199026, 'drop_l5': 0.46061494527734637, 'drop_l6': 0.1605621975527164, 'drop_l7': 0.4311971810561489, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:18:38,544] Trial 346 finished with value: 0.42524009943008423 and parameters: {'drop_l1': 0.6680912700204475, 'drop_l2': 0.3012481319057004, 'drop_l3': 0.3058149162432132, 'drop_l4': 0.6317363895491098, 'drop_l5': 0.467791310105647, 'drop_l6': 0.16081100475039023, 'drop_l7': 0.46711157886824983, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:18:41,411] Trial 347 finished with value: 0.5259428024291992 and parameters: {'drop_l1': 0.6156857688693512, 'drop_l2': 0.3244988872348915, 'drop_l3': 0.3069302802753436, 'drop_l4': 0.6277777312146524, 'drop_l5': 0.4602437038711233, 'drop_l6': 0.16147932289044303, 'drop_l7': 0.4581987941870782, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:18:42,189] Trial 348 finished with value: 0.5485433340072632 and parameters: {'drop_l1': 0.5302684382773456, 'drop_l2': 0.45620677205950516, 'drop_l3': 0.22612844807358368, 'drop_l4': 0.6136501655559707, 'drop_l5': 0.45817968236220863, 'drop_l6': 0.15785463110762857, 'drop_l7': 0.43952610506356204, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:19:07,754] Trial 349 finished with value: 0.4262445569038391 and parameters: {'drop_l1': 0.6424316254154879, 'drop_l2': 0.3326405057608431, 'drop_l3': 0.25978744117952435, 'drop_l4': 0.6167202574923836, 'drop_l5': 0.4700935996616697, 'drop_l6': 0.15796134257214023, 'drop_l7': 0.46678196636043284, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:19:21,853] Trial 350 finished with value: 0.48581480979919434 and parameters: {'drop_l1': 0.17613314145902834, 'drop_l2': 0.335182395912734, 'drop_l3': 0.2645612161677213, 'drop_l4': 0.6083526963396123, 'drop_l5': 0.4347235497255008, 'drop_l6': 0.13425780287783828, 'drop_l7': 0.8942289457383024, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:19:23,962] Trial 352 finished with value: 0.45751819014549255 and parameters: {'drop_l1': 0.1825772216829276, 'drop_l2': 0.3025101683558165, 'drop_l3': 0.2624516182361843, 'drop_l4': 0.6706618822718188, 'drop_l5': 0.43473345617494896, 'drop_l6': 0.1365017834313903, 'drop_l7': 0.41832160064914303, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:19:25,555] Trial 351 finished with value: 0.42232242226600647 and parameters: {'drop_l1': 0.5355380794042646, 'drop_l2': 0.4452797833733467, 'drop_l3': 0.26054681974635113, 'drop_l4': 0.6071262656316824, 'drop_l5': 0.4333695776471792, 'drop_l6': 0.133570522168872, 'drop_l7': 0.4405169225093878, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:19:38,217] Trial 353 finished with value: 0.4324420094490051 and parameters: {'drop_l1': 0.17421878706743757, 'drop_l2': 0.31518224343070866, 'drop_l3': 0.26853880361593013, 'drop_l4': 0.6703701673336887, 'drop_l5': 0.4316275986896176, 'drop_l6': 0.13541794118943773, 'drop_l7': 0.42129505700227754, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:02,217] Trial 354 finished with value: 0.4627997875213623 and parameters: {'drop_l1': 0.7517824880622905, 'drop_l2': 0.3465634566981289, 'drop_l3': 0.272784831821125, 'drop_l4': 0.5588144794243333, 'drop_l5': 0.42332764797547595, 'drop_l6': 0.14019877239697323, 'drop_l7': 0.4143197372145604, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:05,828] Trial 355 finished with value: 0.45873627066612244 and parameters: {'drop_l1': 0.7296762552132078, 'drop_l2': 0.2960349746844178, 'drop_l3': 0.2774751257069316, 'drop_l4': 0.6327724420477241, 'drop_l5': 0.42322806786975026, 'drop_l6': 0.10135651785032349, 'drop_l7': 0.4398821709225619, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:06,857] Trial 356 finished with value: 0.42788630723953247 and parameters: {'drop_l1': 0.20564319076161003, 'drop_l2': 0.31533088285078226, 'drop_l3': 0.6341134224278799, 'drop_l4': 0.6537340808103043, 'drop_l5': 0.42084386855607114, 'drop_l6': 0.11636566420123472, 'drop_l7': 0.45543484688840624, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:14,215] Trial 357 finished with value: 0.4278830587863922 and parameters: {'drop_l1': 0.7242728724987877, 'drop_l2': 0.2875245177887655, 'drop_l3': 0.28811199140894544, 'drop_l4': 0.6503797372196681, 'drop_l5': 0.371373897965295, 'drop_l6': 0.10144456983163865, 'drop_l7': 0.4575312258386449, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:42,097] Trial 358 finished with value: 0.40111008286476135 and parameters: {'drop_l1': 0.20830013150687457, 'drop_l2': 0.2907764890842097, 'drop_l3': 0.35747037610053384, 'drop_l4': 0.6406000678155442, 'drop_l5': 0.4462475790027014, 'drop_l6': 0.1010123813483145, 'drop_l7': 0.454199127969165, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:49,397] Trial 359 finished with value: 0.4470915198326111 and parameters: {'drop_l1': 0.14411546341432266, 'drop_l2': 0.7531349251188941, 'drop_l3': 0.35939344539358953, 'drop_l4': 0.6980985634621998, 'drop_l5': 0.4794083592692462, 'drop_l6': 0.1775786580054453, 'drop_l7': 0.45900930074032925, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:49,412] Trial 360 finished with value: 0.44201746582984924 and parameters: {'drop_l1': 0.8169690228001926, 'drop_l2': 0.3689829739529809, 'drop_l3': 0.35930753280564676, 'drop_l4': 0.6962482403555428, 'drop_l5': 0.37093936368909175, 'drop_l6': 0.164348369757535, 'drop_l7': 0.4898216894738362, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:20:52,442] Trial 361 finished with value: 0.41834908723831177 and parameters: {'drop_l1': 0.13787251414848373, 'drop_l2': 0.3627596537249296, 'drop_l3': 0.3613895871794362, 'drop_l4': 0.6967003619219863, 'drop_l5': 0.48400152281247383, 'drop_l6': 0.18523837528496168, 'drop_l7': 0.4894630733007765, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:21:15,631] Trial 362 finished with value: 0.43195706605911255 and parameters: {'drop_l1': 0.2104517634899999, 'drop_l2': 0.27090759316814883, 'drop_l3': 0.36310854154337174, 'drop_l4': 0.6990584203324234, 'drop_l5': 0.44851176184303543, 'drop_l6': 0.18068430157037949, 'drop_l7': 0.46922602877540204, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:21:34,642] Trial 365 finished with value: 0.42362165451049805 and parameters: {'drop_l1': 0.13311494579080255, 'drop_l2': 0.2587606691715876, 'drop_l3': 0.3885460700701987, 'drop_l4': 0.7161838908859368, 'drop_l5': 0.45054767209472213, 'drop_l6': 0.18347882957639902, 'drop_l7': 0.49614056058566053, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:21:34,657] Trial 364 finished with value: 0.536096453666687 and parameters: {'drop_l1': 0.10011139085698875, 'drop_l2': 0.26714633101386115, 'drop_l3': 0.3318502529453413, 'drop_l4': 0.48465001335080005, 'drop_l5': 0.44725503795336974, 'drop_l6': 0.1826956742700689, 'drop_l7': 0.5010793500602586, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:21:34,674] Trial 363 finished with value: 0.44475942850112915 and parameters: {'drop_l1': 0.20687208876373914, 'drop_l2': 0.25335076447012267, 'drop_l3': 0.43302578660673496, 'drop_l4': 0.6876226328121351, 'drop_l5': 0.44944972170180936, 'drop_l6': 0.1828822481225993, 'drop_l7': 0.47808075970233965, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:21:47,432] Trial 366 finished with value: 0.5255557298660278 and parameters: {'drop_l1': 0.13769707129545414, 'drop_l2': 0.3609036391385589, 'drop_l3': 0.38309561064490033, 'drop_l4': 0.7207247818559476, 'drop_l5': 0.500613739675721, 'drop_l6': 0.1528453687293942, 'drop_l7': 0.5075889919761509, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:22:17,776] Trial 367 finished with value: 0.4286176562309265 and parameters: {'drop_l1': 0.11503085504041809, 'drop_l2': 0.3830470254566865, 'drop_l3': 0.34801323417491964, 'drop_l4': 0.7376397615384885, 'drop_l5': 0.49729974227452084, 'drop_l6': 0.1535485787153581, 'drop_l7': 0.49646609110422524, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:22:19,273] Trial 368 finished with value: 0.4323485791683197 and parameters: {'drop_l1': 0.1434360658537406, 'drop_l2': 0.3545737999954853, 'drop_l3': 0.34973924165635084, 'drop_l4': 0.3360401822576943, 'drop_l5': 0.4998802025398821, 'drop_l6': 0.15031804991836745, 'drop_l7': 0.4904164321918275, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:22:19,286] Trial 369 finished with value: 0.4256429672241211 and parameters: {'drop_l1': 0.14946698544969816, 'drop_l2': 0.38495346563360344, 'drop_l3': 0.34296529850786206, 'drop_l4': 0.33501492767253793, 'drop_l5': 0.4830133365914821, 'drop_l6': 0.15592472573970378, 'drop_l7': 0.4946438072335859, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:22:24,085] Trial 370 finished with value: 0.4430217742919922 and parameters: {'drop_l1': 0.11735371840303137, 'drop_l2': 0.37901175710299934, 'drop_l3': 0.34732081380744934, 'drop_l4': 0.6804711499114254, 'drop_l5': 0.478847992903525, 'drop_l6': 0.11519908194441854, 'drop_l7': 0.47225272432557386, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:01,212] Trial 371 finished with value: 0.45617571473121643 and parameters: {'drop_l1': 0.10024577595904241, 'drop_l2': 0.342558797216175, 'drop_l3': 0.343865268215792, 'drop_l4': 0.34693831441629186, 'drop_l5': 0.48076953970829867, 'drop_l6': 0.11365924177167867, 'drop_l7': 0.4729004459787455, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:02,915] Trial 373 finished with value: 0.4292673170566559 and parameters: {'drop_l1': 0.10013900708550257, 'drop_l2': 0.33995298943516683, 'drop_l3': 0.3287114618767046, 'drop_l4': 0.5861543360023552, 'drop_l5': 0.4746565595049077, 'drop_l6': 0.10036789832396051, 'drop_l7': 0.4760111636577075, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:03,006] Trial 372 finished with value: 0.43120527267456055 and parameters: {'drop_l1': 0.10088033384784732, 'drop_l2': 0.33680320114124, 'drop_l3': 0.32346372364384657, 'drop_l4': 0.6788937614833085, 'drop_l5': 0.4758959020258351, 'drop_l6': 0.11932313820251836, 'drop_l7': 0.46932204676826317, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:04,537] Trial 374 finished with value: 0.4275168478488922 and parameters: {'drop_l1': 0.1948046604258662, 'drop_l2': 0.3492225635859195, 'drop_l3': 0.33106139062643275, 'drop_l4': 0.6294244217812657, 'drop_l5': 0.4158692447640863, 'drop_l6': 0.12092657266712184, 'drop_l7': 0.4480837692668165, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:44,422] Trial 375 finished with value: 0.42305970191955566 and parameters: {'drop_l1': 0.1632578144683391, 'drop_l2': 0.33998372750498773, 'drop_l3': 0.3303799586392619, 'drop_l4': 0.6385005115054289, 'drop_l5': 0.4139338842184647, 'drop_l6': 0.1672106862958449, 'drop_l7': 0.44915310708048406, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:46,633] Trial 377 finished with value: 0.4115733206272125 and parameters: {'drop_l1': 0.19468740570226592, 'drop_l2': 0.367338114367522, 'drop_l3': 0.19957571045812691, 'drop_l4': 0.627526261777591, 'drop_l5': 0.41451680895699106, 'drop_l6': 0.17004233681166622, 'drop_l7': 0.4466122507565998, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:46,769] Trial 378 finished with value: 0.44840383529663086 and parameters: {'drop_l1': 0.1614180714330143, 'drop_l2': 0.28902575899890753, 'drop_l3': 0.13665915931962327, 'drop_l4': 0.7122910061857401, 'drop_l5': 0.40003515581088284, 'drop_l6': 0.17020622967063928, 'drop_l7': 0.44696074165717986, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:23:46,809] Trial 376 finished with value: 0.44680142402648926 and parameters: {'drop_l1': 0.16734945464263928, 'drop_l2': 0.3006649054279677, 'drop_l3': 0.36028114976545317, 'drop_l4': 0.6297564617990156, 'drop_l5': 0.409935887601284, 'drop_l6': 0.169359703620361, 'drop_l7': 0.2955221282844578, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:24:26,766] Trial 379 finished with value: 0.4374226927757263 and parameters: {'drop_l1': 0.12882261632963496, 'drop_l2': 0.5510921528165228, 'drop_l3': 0.3695004645668119, 'drop_l4': 0.7055873696763822, 'drop_l5': 0.440427245669751, 'drop_l6': 0.1429133419525139, 'drop_l7': 0.2994632356276803, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:24:29,990] Trial 380 finished with value: 0.42489689588546753 and parameters: {'drop_l1': 0.18706021632506992, 'drop_l2': 0.36372070059652967, 'drop_l3': 0.36041497862529104, 'drop_l4': 0.6245740063649295, 'drop_l5': 0.3971708885279007, 'drop_l6': 0.17273424066225776, 'drop_l7': 0.31990705906143846, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:24:30,116] Trial 381 finished with value: 0.4508783221244812 and parameters: {'drop_l1': 0.18188830648148152, 'drop_l2': 0.37121929493277395, 'drop_l3': 0.2972034491391509, 'drop_l4': 0.6596698942584145, 'drop_l5': 0.43195145682362396, 'drop_l6': 0.20439646872241574, 'drop_l7': 0.4338987276472271, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:24:30,175] Trial 382 finished with value: 0.41240745782852173 and parameters: {'drop_l1': 0.18546537078382933, 'drop_l2': 0.36658314458265007, 'drop_l3': 0.15451226125426548, 'drop_l4': 0.6604959930595773, 'drop_l5': 0.4364185496976943, 'drop_l6': 0.2027651260155997, 'drop_l7': 0.4053968181953563, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:07,058] Trial 383 finished with value: 0.6661809682846069 and parameters: {'drop_l1': 0.19048201171488277, 'drop_l2': 0.3622306478485669, 'drop_l3': 0.2997551824327767, 'drop_l4': 0.662231619858035, 'drop_l5': 0.3939313193478199, 'drop_l6': 0.20493578690644448, 'drop_l7': 0.4058048019267144, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:11,566] Trial 386 finished with value: 0.5415681004524231 and parameters: {'drop_l1': 0.199373938182521, 'drop_l2': 0.39475907316899866, 'drop_l3': 0.156370859564081, 'drop_l4': 0.6420160348691899, 'drop_l5': 0.4466088212850337, 'drop_l6': 0.21465301363217965, 'drop_l7': 0.40939908839722194, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:11,570] Trial 385 finished with value: 0.4323541522026062 and parameters: {'drop_l1': 0.1479186436857331, 'drop_l2': 0.35713550575414443, 'drop_l3': 0.16111227909407588, 'drop_l4': 0.48433044177975976, 'drop_l5': 0.44409700977833444, 'drop_l6': 0.3134868760645948, 'drop_l7': 0.4065695666647611, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:11,609] Trial 384 finished with value: 0.6012663841247559 and parameters: {'drop_l1': 0.19598402898112027, 'drop_l2': 0.420532892322915, 'drop_l3': 0.19159254782189036, 'drop_l4': 0.6613021204664253, 'drop_l5': 0.42779510033885326, 'drop_l6': 0.21438356460312005, 'drop_l7': 0.41035418085798975, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:41,128] Trial 387 finished with value: 0.42705678939819336 and parameters: {'drop_l1': 0.20886519891279143, 'drop_l2': 0.4105049687796153, 'drop_l3': 0.20555855223262914, 'drop_l4': 0.6011500241544558, 'drop_l5': 0.44841145253826487, 'drop_l6': 0.23915689323143363, 'drop_l7': 0.407995596682429, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:52,433] Trial 390 finished with value: 0.4740733504295349 and parameters: {'drop_l1': 0.1721443789797349, 'drop_l2': 0.3812110051746415, 'drop_l3': 0.21510048995084452, 'drop_l4': 0.5968144883436818, 'drop_l5': 0.4220799684911595, 'drop_l6': 0.14654037068372508, 'drop_l7': 0.4307957695825598, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:52,479] Trial 388 finished with value: 0.4450375437736511 and parameters: {'drop_l1': 0.17183252530408244, 'drop_l2': 0.4209473255390829, 'drop_l3': 0.11533999184434082, 'drop_l4': 0.6187027014788457, 'drop_l5': 0.4161843664946064, 'drop_l6': 0.1889560718353454, 'drop_l7': 0.4340562838132994, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:25:52,505] Trial 389 finished with value: 0.5086758136749268 and parameters: {'drop_l1': 0.2114650903456573, 'drop_l2': 0.42181481776302693, 'drop_l3': 0.14234475525197915, 'drop_l4': 0.6136580226657331, 'drop_l5': 0.4324798005487249, 'drop_l6': 0.14997630843130658, 'drop_l7': 0.43015459972931047, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:26:10,653] Trial 391 finished with value: 0.4918498396873474 and parameters: {'drop_l1': 0.17989583774852325, 'drop_l2': 0.38177187111954186, 'drop_l3': 0.12135221995850377, 'drop_l4': 0.6406163305864453, 'drop_l5': 0.26233460501943007, 'drop_l6': 0.18638836130321973, 'drop_l7': 0.430930045633264, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:26:33,448] Trial 392 finished with value: 0.4537893831729889 and parameters: {'drop_l1': 0.17387223195862997, 'drop_l2': 0.3148377530894467, 'drop_l3': 0.17353094946660833, 'drop_l4': 0.6475661197602235, 'drop_l5': 0.46017521177097853, 'drop_l6': 0.1673144282494661, 'drop_l7': 0.44889059321330893, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:26:34,183] Trial 393 finished with value: 0.4351162016391754 and parameters: {'drop_l1': 0.215995697661323, 'drop_l2': 0.30953488221725683, 'drop_l3': 0.1640208824875553, 'drop_l4': 0.6382018370159219, 'drop_l5': 0.41670114272229636, 'drop_l6': 0.16621737541831452, 'drop_l7': 0.4532770832313905, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:26:34,684] Trial 394 finished with value: 0.42536669969558716 and parameters: {'drop_l1': 0.24653175956797396, 'drop_l2': 0.31241162140114365, 'drop_l3': 0.11861917109290707, 'drop_l4': 0.7311995056882992, 'drop_l5': 0.45864151100519424, 'drop_l6': 0.16714643985898972, 'drop_l7': 0.45294364705446655, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:26:42,491] Trial 395 finished with value: 0.4253265857696533 and parameters: {'drop_l1': 0.24708358541868064, 'drop_l2': 0.3220865162200052, 'drop_l3': 0.1583505008919949, 'drop_l4': 0.37603824472961495, 'drop_l5': 0.4561003960753923, 'drop_l6': 0.16301072139494535, 'drop_l7': 0.4463510885829377, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:15,396] Trial 396 finished with value: 0.5393881797790527 and parameters: {'drop_l1': 0.15083518124746553, 'drop_l2': 0.32173862030945166, 'drop_l3': 0.14068106624213586, 'drop_l4': 0.7391084216207733, 'drop_l5': 0.4040620418349449, 'drop_l6': 0.23058784508456076, 'drop_l7': 0.46532028837822054, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:15,975] Trial 397 finished with value: 0.4253300130367279 and parameters: {'drop_l1': 0.15036422972176003, 'drop_l2': 0.3368370441089428, 'drop_l3': 0.11217924503938959, 'drop_l4': 0.7341376079417682, 'drop_l5': 0.4373748910826513, 'drop_l6': 0.13759518542272933, 'drop_l7': 0.38864487589810764, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:16,636] Trial 398 finished with value: 0.5150372982025146 and parameters: {'drop_l1': 0.15084301729299476, 'drop_l2': 0.3273185132355621, 'drop_l3': 0.14867981825610233, 'drop_l4': 0.6723000230632996, 'drop_l5': 0.40415329525049587, 'drop_l6': 0.13830154426175922, 'drop_l7': 0.4671939372086445, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:19,536] Trial 399 finished with value: 0.5452386140823364 and parameters: {'drop_l1': 0.15198117572029934, 'drop_l2': 0.3684194836849876, 'drop_l3': 0.13442195288209444, 'drop_l4': 0.6723458821815087, 'drop_l5': 0.434135798024653, 'drop_l6': 0.1379129163431239, 'drop_l7': 0.39152493099900343, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:58,979] Trial 400 finished with value: 0.5417393445968628 and parameters: {'drop_l1': 0.1196020809253995, 'drop_l2': 0.35751831329374073, 'drop_l3': 0.4545422349237259, 'drop_l4': 0.6723824690769951, 'drop_l5': 0.43197135512928986, 'drop_l6': 0.1429496668600017, 'drop_l7': 0.8416289552183731, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:59,744] Trial 402 finished with value: 0.46420422196388245 and parameters: {'drop_l1': 0.11941760198325409, 'drop_l2': 0.3689195237709639, 'drop_l3': 0.1015526933945466, 'drop_l4': 0.6866603420972271, 'drop_l5': 0.4322243444408139, 'drop_l6': 0.19244717545517323, 'drop_l7': 0.4175175058695079, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:59,787] Trial 401 finished with value: 0.4349118173122406 and parameters: {'drop_l1': 0.1196645134271463, 'drop_l2': 0.3660875958786877, 'drop_l3': 0.8982442583888606, 'drop_l4': 0.6700051193135848, 'drop_l5': 0.4347189294803186, 'drop_l6': 0.19629618828813636, 'drop_l7': 0.4861051640567163, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:27:59,843] Trial 403 finished with value: 0.4724271893501282 and parameters: {'drop_l1': 0.12340139156682373, 'drop_l2': 0.3486943285687091, 'drop_l3': 0.1909286787148396, 'drop_l4': 0.6895529173348361, 'drop_l5': 0.4226212934461509, 'drop_l6': 0.1929586509971282, 'drop_l7': 0.42208183397783205, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:28:40,143] Trial 404 finished with value: 0.44093626737594604 and parameters: {'drop_l1': 0.2317106388314088, 'drop_l2': 0.3483396530148565, 'drop_l3': 0.2378453802657444, 'drop_l4': 0.42539765803076013, 'drop_l5': 0.4155691751010287, 'drop_l6': 0.19399488984263316, 'drop_l7': 0.4224627067608071, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:28:43,254] Trial 407 finished with value: 0.46021050214767456 and parameters: {'drop_l1': 0.22262443597658543, 'drop_l2': 0.39386461209012746, 'drop_l3': 0.23057706835563774, 'drop_l4': 0.40914857242222263, 'drop_l5': 0.40957720577847684, 'drop_l6': 0.180565779095671, 'drop_l7': 0.510300927084246, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:28:43,259] Trial 405 finished with value: 0.44331231713294983 and parameters: {'drop_l1': 0.20183948591843623, 'drop_l2': 0.3511229281881126, 'drop_l3': 0.5541971924278344, 'drop_l4': 0.6532151799782135, 'drop_l5': 0.4154316811433676, 'drop_l6': 0.17571759377247242, 'drop_l7': 0.5074797388300691, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:28:43,278] Trial 406 finished with value: 0.5061682462692261 and parameters: {'drop_l1': 0.2013246051634218, 'drop_l2': 0.3507893738908183, 'drop_l3': 0.23166345720843437, 'drop_l4': 0.4067608977019879, 'drop_l5': 0.4111916353566217, 'drop_l6': 0.43813587480877253, 'drop_l7': 0.5079352985859642, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:29:20,495] Trial 408 finished with value: 0.4362098276615143 and parameters: {'drop_l1': 0.19161426623410543, 'drop_l2': 0.2837208898630307, 'drop_l3': 0.5015533138022736, 'drop_l4': 0.4005180725320353, 'drop_l5': 0.2914124982593475, 'drop_l6': 0.17787121352258284, 'drop_l7': 0.5099687550437244, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:29:25,723] Trial 409 finished with value: 0.4403809905052185 and parameters: {'drop_l1': 0.5808491250431426, 'drop_l2': 0.2902294394432069, 'drop_l3': 0.3105947337645408, 'drop_l4': 0.7052960767066025, 'drop_l5': 0.39157733698152747, 'drop_l6': 0.15511944293466132, 'drop_l7': 0.43910975878610764, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:29:25,888] Trial 410 finished with value: 0.4451669156551361 and parameters: {'drop_l1': 0.1893659811343077, 'drop_l2': 0.3279351728133065, 'drop_l3': 0.40723175037183873, 'drop_l4': 0.7049309122424774, 'drop_l5': 0.3803044185926975, 'drop_l6': 0.15136831307714735, 'drop_l7': 0.44102787447260733, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:29:25,891] Trial 411 finished with value: 0.6048418283462524 and parameters: {'drop_l1': 0.13325680044689955, 'drop_l2': 0.3021475237217213, 'drop_l3': 0.3139533214969132, 'drop_l4': 0.7037783842349223, 'drop_l5': 0.37648383543951536, 'drop_l6': 0.8948308902001552, 'drop_l7': 0.4394383227275088, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:29:57,374] Trial 412 finished with value: 0.40765661001205444 and parameters: {'drop_l1': 0.1338726896458145, 'drop_l2': 0.29612334629101844, 'drop_l3': 0.30968579448027445, 'drop_l4': 0.7017998258017804, 'drop_l5': 0.38130257460459605, 'drop_l6': 0.15235837181353007, 'drop_l7': 0.43997340193527146, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:07,119] Trial 413 finished with value: 0.4289742112159729 and parameters: {'drop_l1': 0.13501535874589002, 'drop_l2': 0.3279498210217008, 'drop_l3': 0.3118225350539271, 'drop_l4': 0.654993177421795, 'drop_l5': 0.22304039476582432, 'drop_l6': 0.10038131188627131, 'drop_l7': 0.48156271804988643, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:08,281] Trial 415 finished with value: 0.4168001711368561 and parameters: {'drop_l1': 0.2615399208106817, 'drop_l2': 0.37648710440617, 'drop_l3': 0.2953750086018996, 'drop_l4': 0.6537342195245486, 'drop_l5': 0.45039599214980214, 'drop_l6': 0.12888275039908173, 'drop_l7': 0.4830581997098288, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:08,299] Trial 414 finished with value: 0.4284440875053406 and parameters: {'drop_l1': 0.1650318006829592, 'drop_l2': 0.382694490519813, 'drop_l3': 0.28742659433466533, 'drop_l4': 0.6309427266051113, 'drop_l5': 0.4666842815530161, 'drop_l6': 0.1029686882014419, 'drop_l7': 0.6230817083904897, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:28,744] Trial 416 finished with value: 0.43207067251205444 and parameters: {'drop_l1': 0.2873872087897832, 'drop_l2': 0.2797063865819004, 'drop_l3': 0.2955420670335021, 'drop_l4': 0.7242137059371943, 'drop_l5': 0.46661283816465715, 'drop_l6': 0.21586883462892487, 'drop_l7': 0.47611829668259625, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:48,249] Trial 417 finished with value: 0.43157505989074707 and parameters: {'drop_l1': 0.1683943446826488, 'drop_l2': 0.38332014890474075, 'drop_l3': 0.2958447347894536, 'drop_l4': 0.29075624532944755, 'drop_l5': 0.36675408422930317, 'drop_l6': 0.21585003999008945, 'drop_l7': 0.4657247626677672, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:50,038] Trial 419 finished with value: 0.4532112181186676 and parameters: {'drop_l1': 0.10012025985230304, 'drop_l2': 0.3835260611923822, 'drop_l3': 0.29561179427325396, 'drop_l4': 0.7234213933412976, 'drop_l5': 0.3710356904839777, 'drop_l6': 0.21311229597287754, 'drop_l7': 0.4879660734481768, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:30:50,600] Trial 418 finished with value: 0.46840834617614746 and parameters: {'drop_l1': 0.16654128625965964, 'drop_l2': 0.3904696452220245, 'drop_l3': 0.28926231498467087, 'drop_l4': 0.7134674321390485, 'drop_l5': 0.4467862369950867, 'drop_l6': 0.21138758641695637, 'drop_l7': 0.4586551256092155, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:31:01,350] Trial 420 finished with value: 0.43000203371047974 and parameters: {'drop_l1': 0.2301477488132563, 'drop_l2': 0.3948483195619866, 'drop_l3': 0.293656958570335, 'drop_l4': 0.6896554763687457, 'drop_l5': 0.37050774729693065, 'drop_l6': 0.15820860152707547, 'drop_l7': 0.25844455220192564, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:31:29,516] Trial 421 finished with value: 0.5208061337471008 and parameters: {'drop_l1': 0.22047420317794506, 'drop_l2': 0.39702009491424683, 'drop_l3': 0.28185223185903524, 'drop_l4': 0.6897880574071015, 'drop_l5': 0.3960857191880128, 'drop_l6': 0.15896638712822184, 'drop_l7': 0.45989526690291394, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:31:31,837] Trial 422 finished with value: 0.45560869574546814 and parameters: {'drop_l1': 0.23476530921073072, 'drop_l2': 0.3717782059881118, 'drop_l3': 0.31444257272426324, 'drop_l4': 0.690955048233352, 'drop_l5': 0.44730444945245434, 'drop_l6': 0.12792469699947373, 'drop_l7': 0.45821476579846354, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:31:32,953] Trial 423 finished with value: 0.49442315101623535 and parameters: {'drop_l1': 0.2278416377912987, 'drop_l2': 0.36365367155836337, 'drop_l3': 0.2690972244818882, 'drop_l4': 0.6904491338966467, 'drop_l5': 0.39489481844806956, 'drop_l6': 0.15820130474571947, 'drop_l7': 0.4488221032159886, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:31:37,460] Trial 424 finished with value: 0.5197828412055969 and parameters: {'drop_l1': 0.25858160964208143, 'drop_l2': 0.6090483436073149, 'drop_l3': 0.27816852666272457, 'drop_l4': 0.6853945187777102, 'drop_l5': 0.3993311222965988, 'drop_l6': 0.15947784743819154, 'drop_l7': 0.4545344610650114, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:10,820] Trial 425 finished with value: 0.5863226056098938 and parameters: {'drop_l1': 0.2644125794111134, 'drop_l2': 0.3595178365790431, 'drop_l3': 0.25431038374730847, 'drop_l4': 0.7470377085741995, 'drop_l5': 0.4501284860941319, 'drop_l6': 0.12758896559144756, 'drop_l7': 0.4907275597203996, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:15,800] Trial 426 finished with value: 0.44628387689590454 and parameters: {'drop_l1': 0.25764826881837344, 'drop_l2': 0.34388253514303124, 'drop_l3': 0.3914360901720328, 'drop_l4': 0.7463541860872946, 'drop_l5': 0.3950189435096254, 'drop_l6': 0.17840471950030296, 'drop_l7': 0.4833067274802331, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:15,934] Trial 427 finished with value: 0.5369935631752014 and parameters: {'drop_l1': 0.2592098746609214, 'drop_l2': 0.4741089501891813, 'drop_l3': 0.37871798658077027, 'drop_l4': 0.6637466653933676, 'drop_l5': 0.450201858189804, 'drop_l6': 0.17322159479526608, 'drop_l7': 0.4914342803094033, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:16,919] Trial 428 finished with value: 0.5708105564117432 and parameters: {'drop_l1': 0.18438474324208634, 'drop_l2': 0.2468065460880998, 'drop_l3': 0.24822177871162907, 'drop_l4': 0.6610210016006455, 'drop_l5': 0.4477584725295774, 'drop_l6': 0.7684945613783458, 'drop_l7': 0.49013059958209465, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:49,407] Trial 429 finished with value: 0.41889849305152893 and parameters: {'drop_l1': 0.1874205006879324, 'drop_l2': 0.3325491518371975, 'drop_l3': 0.3913674145139152, 'drop_l4': 0.637552778059348, 'drop_l5': 0.43813142902295804, 'drop_l6': 0.17520620634415218, 'drop_l7': 0.43610138393571135, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:58,450] Trial 432 finished with value: 0.4409882426261902 and parameters: {'drop_l1': 0.14144901346323013, 'drop_l2': 0.2966019333472627, 'drop_l3': 0.3445671709005885, 'drop_l4': 0.6388442577010521, 'drop_l5': 0.42800157898410296, 'drop_l6': 0.23519450499149397, 'drop_l7': 0.421751292765421, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:58,541] Trial 431 finished with value: 0.5007221698760986 and parameters: {'drop_l1': 0.1435953225172456, 'drop_l2': 0.24379379407950263, 'drop_l3': 0.535565149701625, 'drop_l4': 0.6397666363647072, 'drop_l5': 0.42597618971849255, 'drop_l6': 0.2330161749139024, 'drop_l7': 0.4000664025634955, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:32:58,613] Trial 430 finished with value: 0.5220632553100586 and parameters: {'drop_l1': 0.14035973768632415, 'drop_l2': 0.33704515424094983, 'drop_l3': 0.3430260235789678, 'drop_l4': 0.6624019167254609, 'drop_l5': 0.42844139162532263, 'drop_l6': 0.184622143373586, 'drop_l7': 0.43534192359821966, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:33:20,176] Trial 433 finished with value: 0.44026979804039 and parameters: {'drop_l1': 0.1408772040245469, 'drop_l2': 0.2991741045946653, 'drop_l3': 0.3443550311790805, 'drop_l4': 0.6223078472602693, 'drop_l5': 0.42160340636824317, 'drop_l6': 0.14487053735241723, 'drop_l7': 0.41498768157587984, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:33:38,638] Trial 434 finished with value: 0.456306517124176 and parameters: {'drop_l1': 0.21075424438045212, 'drop_l2': 0.3690289594931063, 'drop_l3': 0.32382927473763207, 'drop_l4': 0.6207002229532194, 'drop_l5': 0.4224051645048536, 'drop_l6': 0.19720411472194332, 'drop_l7': 0.44461517459019356, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:33:39,550] Trial 436 finished with value: 0.4641249179840088 and parameters: {'drop_l1': 0.20621064366450803, 'drop_l2': 0.37543397626813935, 'drop_l3': 0.3169986075974454, 'drop_l4': 0.5777470707897513, 'drop_l5': 0.4661891789971747, 'drop_l6': 0.20150090570768858, 'drop_l7': 0.4689165596576231, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:33:39,599] Trial 435 finished with value: 0.45231181383132935 and parameters: {'drop_l1': 0.21380378670324568, 'drop_l2': 0.2736219978544459, 'drop_l3': 0.32065611976468295, 'drop_l4': 0.6164593307847741, 'drop_l5': 0.4099372896368196, 'drop_l6': 0.2007162580329104, 'drop_l7': 0.37806012287754087, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:33:50,902] Trial 437 finished with value: 0.47732624411582947 and parameters: {'drop_l1': 0.20697323060088577, 'drop_l2': 0.40816572003034174, 'drop_l3': 0.3211186095253385, 'drop_l4': 0.7092206866293982, 'drop_l5': 0.4032237649198224, 'drop_l6': 0.12077792948538255, 'drop_l7': 0.8787178762977209, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:34:21,212] Trial 438 finished with value: 0.4319342076778412 and parameters: {'drop_l1': 0.10698846499520963, 'drop_l2': 0.35163714264231677, 'drop_l3': 0.30664169040815015, 'drop_l4': 0.5829891158759009, 'drop_l5': 0.4651850418198259, 'drop_l6': 0.12762578684663914, 'drop_l7': 0.4708847476141612, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:34:22,987] Trial 440 finished with value: 0.47283631563186646 and parameters: {'drop_l1': 0.10975701977186925, 'drop_l2': 0.35051582400338666, 'drop_l3': 0.12941816446939497, 'drop_l4': 0.6758767873346778, 'drop_l5': 0.3821686609385846, 'drop_l6': 0.12843036273361377, 'drop_l7': 0.4065394414326126, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:34:23,082] Trial 439 finished with value: 0.5674412846565247 and parameters: {'drop_l1': 0.11079475834795904, 'drop_l2': 0.41000354738999295, 'drop_l3': 0.8761862146513871, 'drop_l4': 0.7157410833820599, 'drop_l5': 0.4899671257903682, 'drop_l6': 0.12399645467994191, 'drop_l7': 0.4251700072059523, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:34:27,959] Trial 441 finished with value: 0.5657482147216797 and parameters: {'drop_l1': 0.10836243540451698, 'drop_l2': 0.3437122739790697, 'drop_l3': 0.8739089055570075, 'drop_l4': 0.6780878820474607, 'drop_l5': 0.5145640680392544, 'drop_l6': 0.14561840505677534, 'drop_l7': 0.4716850868333585, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:04,172] Trial 442 finished with value: 0.5138652324676514 and parameters: {'drop_l1': 0.11067489178214159, 'drop_l2': 0.3451027718054334, 'drop_l3': 0.10206278431544935, 'drop_l4': 0.6761423486314913, 'drop_l5': 0.3799757101425988, 'drop_l6': 0.5630805039427351, 'drop_l7': 0.39819998947808344, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:07,629] Trial 443 finished with value: 0.4168473780155182 and parameters: {'drop_l1': 0.24287166438121083, 'drop_l2': 0.3304973073862173, 'drop_l3': 0.1968880292814411, 'drop_l4': 0.20943005955914973, 'drop_l5': 0.5095715068714151, 'drop_l6': 0.14665301237936065, 'drop_l7': 0.5133415421370433, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:08,215] Trial 444 finished with value: 0.43981823325157166 and parameters: {'drop_l1': 0.16067008626629195, 'drop_l2': 0.32899858009412836, 'drop_l3': 0.27309528913416237, 'drop_l4': 0.6529110913661429, 'drop_l5': 0.4478773325651852, 'drop_l6': 0.146679006964637, 'drop_l7': 0.5264933396090155, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:08,817] Trial 445 finished with value: 0.4446136951446533 and parameters: {'drop_l1': 0.1610585525933756, 'drop_l2': 0.30769137002246877, 'drop_l3': 0.27277455440444287, 'drop_l4': 0.6508704938694405, 'drop_l5': 0.4427557446040615, 'drop_l6': 0.1695321771222426, 'drop_l7': 0.5211430464853591, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:47,633] Trial 446 finished with value: 0.43002963066101074 and parameters: {'drop_l1': 0.1582544944497452, 'drop_l2': 0.31088667971899847, 'drop_l3': 0.271491095776799, 'drop_l4': 0.6544990228902204, 'drop_l5': 0.4490383782245824, 'drop_l6': 0.16710439884030875, 'drop_l7': 0.4495340656435742, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:50,559] Trial 447 finished with value: 0.42077574133872986 and parameters: {'drop_l1': 0.24420756812157213, 'drop_l2': 0.3666362792779553, 'drop_l3': 0.18525816383919702, 'drop_l4': 0.6478445796960605, 'drop_l5': 0.5035663249451343, 'drop_l6': 0.1642059261731296, 'drop_l7': 0.5154540613902799, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:51,623] Trial 449 finished with value: 0.4457334578037262 and parameters: {'drop_l1': 0.2410050952614454, 'drop_l2': 0.3651689969712634, 'drop_l3': 0.1925165549182924, 'drop_l4': 0.26285116187907137, 'drop_l5': 0.52409200708392, 'drop_l6': 0.17781898315631928, 'drop_l7': 0.5496571796775562, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:35:52,227] Trial 448 finished with value: 0.4279276728630066 and parameters: {'drop_l1': 0.23782855697913216, 'drop_l2': 0.3636615727244984, 'drop_l3': 0.2307186690785744, 'drop_l4': 0.18285157552442521, 'drop_l5': 0.5091434220289704, 'drop_l6': 0.16334478230374683, 'drop_l7': 0.49884355104428285, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:36:26,859] Trial 450 finished with value: 0.4216715693473816 and parameters: {'drop_l1': 0.24172881357985623, 'drop_l2': 0.37497739121424895, 'drop_l3': 0.22592310954217226, 'drop_l4': 0.14083989920670825, 'drop_l5': 0.47825584439069646, 'drop_l6': 0.18548129515196293, 'drop_l7': 0.5042905425409785, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:36:31,783] Trial 453 finished with value: 0.43940576910972595 and parameters: {'drop_l1': 0.279055734129643, 'drop_l2': 0.33353989448430876, 'drop_l3': 0.17141060891028098, 'drop_l4': 0.5561787166688349, 'drop_l5': 0.48026015334524863, 'drop_l6': 0.18913588850705343, 'drop_l7': 0.5016483683855111, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:36:31,836] Trial 451 finished with value: 0.4401993155479431 and parameters: {'drop_l1': 0.2373539165319782, 'drop_l2': 0.3313958653831496, 'drop_l3': 0.2231498517183339, 'drop_l4': 0.2692631180891626, 'drop_l5': 0.5152530734701621, 'drop_l6': 0.18611608579345595, 'drop_l7': 0.5437325025698038, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:36:31,888] Trial 452 finished with value: 0.45116907358169556 and parameters: {'drop_l1': 0.2936649700214527, 'drop_l2': 0.33029489056951256, 'drop_l3': 0.20923960330351254, 'drop_l4': 0.6976730747178197, 'drop_l5': 0.4897191801256857, 'drop_l6': 0.18823013234773045, 'drop_l7': 0.5386453873708299, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:36:58,475] Trial 454 finished with value: 0.6170097589492798 and parameters: {'drop_l1': 0.2753813655588305, 'drop_l2': 0.34165708845272585, 'drop_l3': 0.20615386193161175, 'drop_l4': 0.18538997439586205, 'drop_l5': 0.49150009106781045, 'drop_l6': 0.8577983111327374, 'drop_l7': 0.5405125501643422, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:12,059] Trial 455 finished with value: 0.6175329685211182 and parameters: {'drop_l1': 0.18261369816543105, 'drop_l2': 0.3323464159255945, 'drop_l3': 0.20836224494061414, 'drop_l4': 0.20540625289490588, 'drop_l5': 0.4630507188808683, 'drop_l6': 0.8736637553443194, 'drop_l7': 0.594902173970574, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:12,944] Trial 457 finished with value: 0.49117928743362427 and parameters: {'drop_l1': 0.22090531082855147, 'drop_l2': 0.3523962695848979, 'drop_l3': 0.132801045768428, 'drop_l4': 0.6301566651912718, 'drop_l5': 0.46145114859027836, 'drop_l6': 0.684031302787204, 'drop_l7': 0.42563803492396196, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:12,982] Trial 456 finished with value: 0.5835916996002197 and parameters: {'drop_l1': 0.18388840285465566, 'drop_l2': 0.35577887892378895, 'drop_l3': 0.35800760525548475, 'drop_l4': 0.6050730392305306, 'drop_l5': 0.46828126841990914, 'drop_l6': 0.8729730544568142, 'drop_l7': 0.6001859589615584, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:26,469] Trial 458 finished with value: 0.42218175530433655 and parameters: {'drop_l1': 0.19294451946808538, 'drop_l2': 0.435606887820824, 'drop_l3': 0.36465428673334055, 'drop_l4': 0.6704879484088729, 'drop_l5': 0.4705742536282638, 'drop_l6': 0.1453896676064874, 'drop_l7': 0.4386652760737444, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:52,433] Trial 459 finished with value: 0.46698102355003357 and parameters: {'drop_l1': 0.21447309732226602, 'drop_l2': 0.3533337952627784, 'drop_l3': 0.35896371814125305, 'drop_l4': 0.10602205118134148, 'drop_l5': 0.3529296185268349, 'drop_l6': 0.5132365333456814, 'drop_l7': 0.42140850054880447, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:53,312] Trial 460 finished with value: 0.42786717414855957 and parameters: {'drop_l1': 0.19063570217034137, 'drop_l2': 0.3865164777945383, 'drop_l3': 0.15499794731336933, 'drop_l4': 0.10984729196143188, 'drop_l5': 0.4370622920380727, 'drop_l6': 0.1460506900485569, 'drop_l7': 0.4498540471550711, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:53,553] Trial 461 finished with value: 0.41057682037353516 and parameters: {'drop_l1': 0.19750043255306365, 'drop_l2': 0.3834209769063247, 'drop_l3': 0.2515698675159531, 'drop_l4': 0.6728695634795367, 'drop_l5': 0.35638172197498563, 'drop_l6': 0.14928658753247137, 'drop_l7': 0.31476840922614335, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:37:58,340] Trial 462 finished with value: 0.465200811624527 and parameters: {'drop_l1': 0.21766992810601812, 'drop_l2': 0.38625457173544075, 'drop_l3': 0.2554734678766989, 'drop_l4': 0.6652342340027881, 'drop_l5': 0.43760029859866006, 'drop_l6': 0.11031965194960952, 'drop_l7': 0.28146728533622734, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:38:31,921] Trial 463 finished with value: 0.419135719537735 and parameters: {'drop_l1': 0.1722949746625223, 'drop_l2': 0.3927368320950192, 'drop_l3': 0.15193196257781702, 'drop_l4': 0.6681450193365872, 'drop_l5': 0.43585527420188375, 'drop_l6': 0.11121288999121942, 'drop_l7': 0.45593599097307214, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:38:34,357] Trial 465 finished with value: 0.8232042193412781 and parameters: {'drop_l1': 0.25093482382911203, 'drop_l2': 0.37706396881557, 'drop_l3': 0.24556459575942186, 'drop_l4': 0.6940742345727712, 'drop_l5': 0.35459158158205945, 'drop_l6': 0.10187687613275512, 'drop_l7': 0.30301089403997694, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:38:34,376] Trial 464 finished with value: 0.4389236867427826 and parameters: {'drop_l1': 0.2230359370702173, 'drop_l2': 0.3197934412677756, 'drop_l3': 0.33262550584658884, 'drop_l4': 0.6682404942440239, 'drop_l5': 0.31638247644348594, 'drop_l6': 0.11272418493599777, 'drop_l7': 0.3988252874037367, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:38:35,736] Trial 466 finished with value: 0.43081673979759216 and parameters: {'drop_l1': 0.254403908914804, 'drop_l2': 0.36948927202869497, 'drop_l3': 0.30044838903892856, 'drop_l4': 0.6991175699623269, 'drop_l5': 0.31774764291374663, 'drop_l6': 0.11103788878623809, 'drop_l7': 0.322203106156675, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:10,165] Trial 467 finished with value: 0.4979289472103119 and parameters: {'drop_l1': 0.24791426885957735, 'drop_l2': 0.37768930995041916, 'drop_l3': 0.24902915878872903, 'drop_l4': 0.6972633057942024, 'drop_l5': 0.31048712641217346, 'drop_l6': 0.15474506150743522, 'drop_l7': 0.3275977146401939, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:13,213] Trial 469 finished with value: 0.4397996962070465 and parameters: {'drop_l1': 0.29609334506058405, 'drop_l2': 0.3692957702231732, 'drop_l3': 0.19631074117732739, 'drop_l4': 0.6411900451603258, 'drop_l5': 0.3706884756605598, 'drop_l6': 0.15365195791578865, 'drop_l7': 0.5204195793393182, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:14,562] Trial 468 finished with value: 0.4566723704338074 and parameters: {'drop_l1': 0.2967974226325829, 'drop_l2': 0.4005896825045863, 'drop_l3': 0.19334145094591296, 'drop_l4': 0.6405865060672048, 'drop_l5': 0.3183441758363807, 'drop_l6': 0.15084611328767858, 'drop_l7': 0.28492776003409626, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:15,079] Trial 470 finished with value: 0.43847036361694336 and parameters: {'drop_l1': 0.20274709375453154, 'drop_l2': 0.3584294018810462, 'drop_l3': 0.25298067381070005, 'drop_l4': 0.6315381368724633, 'drop_l5': 0.3250619371145311, 'drop_l6': 0.15202950533773368, 'drop_l7': 0.3296019838970164, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:48,384] Trial 471 finished with value: 0.4525829255580902 and parameters: {'drop_l1': 0.2948566081411987, 'drop_l2': 0.3984889638724805, 'drop_l3': 0.1814590882978748, 'drop_l4': 0.641547912879683, 'drop_l5': 0.3678757238586796, 'drop_l6': 0.2530027837112057, 'drop_l7': 0.29063473379021343, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:53,275] Trial 472 finished with value: 0.41555771231651306 and parameters: {'drop_l1': 0.12846331095443214, 'drop_l2': 0.4100983862025234, 'drop_l3': 0.33855598698019684, 'drop_l4': 0.6294814443582338, 'drop_l5': 0.3318972713514065, 'drop_l6': 0.13456544787741434, 'drop_l7': 0.3257154504852547, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:55,192] Trial 474 finished with value: 0.475679486989975 and parameters: {'drop_l1': 0.2726371940543989, 'drop_l2': 0.4071908698329756, 'drop_l3': 0.1680503852872135, 'drop_l4': 0.23812585040962145, 'drop_l5': 0.3564735960837995, 'drop_l6': 0.1362123046195068, 'drop_l7': 0.2958791866940871, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:39:55,297] Trial 473 finished with value: 0.516105055809021 and parameters: {'drop_l1': 0.13303857127804303, 'drop_l2': 0.4118610038406917, 'drop_l3': 0.33629699269061886, 'drop_l4': 0.6833473280410646, 'drop_l5': 0.3905157029343048, 'drop_l6': 0.13131439874119266, 'drop_l7': 0.30382038030871095, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:40:20,941] Trial 475 finished with value: 0.42850303649902344 and parameters: {'drop_l1': 0.1309842012055543, 'drop_l2': 0.3115196984113495, 'drop_l3': 0.33736080855956524, 'drop_l4': 0.6850478927015412, 'drop_l5': 0.34878582070306374, 'drop_l6': 0.13270848820553538, 'drop_l7': 0.5730263861561514, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:40:34,019] Trial 476 finished with value: 0.5148202180862427 and parameters: {'drop_l1': 0.2697095309351088, 'drop_l2': 0.407997903288015, 'drop_l3': 0.28868318621636685, 'drop_l4': 0.6098311383344172, 'drop_l5': 0.3514279861692705, 'drop_l6': 0.13229754730050322, 'drop_l7': 0.31254995149466314, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:40:36,243] Trial 478 finished with value: 0.442681223154068 and parameters: {'drop_l1': 0.22783297565241908, 'drop_l2': 0.3108227752388216, 'drop_l3': 0.30640539379323367, 'drop_l4': 0.6274860542011494, 'drop_l5': 0.32889351835918024, 'drop_l6': 0.13057069464229276, 'drop_l7': 0.37687003696743004, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:40:36,277] Trial 477 finished with value: 0.4589734971523285 and parameters: {'drop_l1': 0.2291446431899717, 'drop_l2': 0.4299000810020277, 'drop_l3': 0.33913348804739735, 'drop_l4': 0.6060008986957675, 'drop_l5': 0.3288462842204693, 'drop_l6': 0.12663587755265857, 'drop_l7': 0.31048324754516354, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:40:51,066] Trial 479 finished with value: 0.4380277693271637 and parameters: {'drop_l1': 0.200466788851349, 'drop_l2': 0.3885190806141746, 'drop_l3': 0.8969799153377515, 'drop_l4': 0.6322024685125305, 'drop_l5': 0.34736892730822083, 'drop_l6': 0.1270982988215396, 'drop_l7': 0.3397945947203115, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:41:14,180] Trial 480 finished with value: 0.40615448355674744 and parameters: {'drop_l1': 0.19813077749090766, 'drop_l2': 0.3822002315499653, 'drop_l3': 0.1287881712091788, 'drop_l4': 0.627859579361195, 'drop_l5': 0.29094908157717525, 'drop_l6': 0.11731730921301947, 'drop_l7': 0.3810721776644347, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:41:17,653] Trial 482 finished with value: 0.4414779543876648 and parameters: {'drop_l1': 0.20409735240397706, 'drop_l2': 0.8854221029852072, 'drop_l3': 0.10063617625701374, 'drop_l4': 0.6553179436885938, 'drop_l5': 0.3353723581968366, 'drop_l6': 0.1005150579155517, 'drop_l7': 0.34685911583074047, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:41:18,058] Trial 481 finished with value: 0.4177308678627014 and parameters: {'drop_l1': 0.19702374532203165, 'drop_l2': 0.3917275819929771, 'drop_l3': 0.8885031111444823, 'drop_l4': 0.61939137027409, 'drop_l5': 0.29036841204418484, 'drop_l6': 0.1024661883212438, 'drop_l7': 0.3524808728656423, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:41:26,172] Trial 483 finished with value: 0.418241024017334 and parameters: {'drop_l1': 0.17027707058446453, 'drop_l2': 0.38482346752680546, 'drop_l3': 0.12040306635658371, 'drop_l4': 0.6536951200227853, 'drop_l5': 0.38846052663311925, 'drop_l6': 0.1434849414881835, 'drop_l7': 0.3238082980159694, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:41:58,825] Trial 484 finished with value: 0.4476259648799896 and parameters: {'drop_l1': 0.20209555097413645, 'drop_l2': 0.38045356602452857, 'drop_l3': 0.12643736071272466, 'drop_l4': 0.6184701469734463, 'drop_l5': 0.3344505967869037, 'drop_l6': 0.11262739575106837, 'drop_l7': 0.35855238829878483, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:03,148] Trial 486 finished with value: 0.4218173027038574 and parameters: {'drop_l1': 0.20787470974910432, 'drop_l2': 0.45001594367771114, 'drop_l3': 0.12668758868417773, 'drop_l4': 0.5911247811473752, 'drop_l5': 0.24380722215231107, 'drop_l6': 0.10258183753953018, 'drop_l7': 0.36670734126386656, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:03,185] Trial 485 finished with value: 0.435279905796051 and parameters: {'drop_l1': 0.17440072991003364, 'drop_l2': 0.3804035041580705, 'drop_l3': 0.12905447098128178, 'drop_l4': 0.5977808989286786, 'drop_l5': 0.29630696034853155, 'drop_l6': 0.11386777033215878, 'drop_l7': 0.3630086761432712, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:06,297] Trial 487 finished with value: 0.5581276416778564 and parameters: {'drop_l1': 0.21578107093087823, 'drop_l2': 0.45479451639669677, 'drop_l3': 0.8960926710850217, 'drop_l4': 0.5937212460727478, 'drop_l5': 0.28051969092073276, 'drop_l6': 0.10140223460759488, 'drop_l7': 0.3480767858889116, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:39,254] Trial 488 finished with value: 0.5726785659790039 and parameters: {'drop_l1': 0.19385448418006584, 'drop_l2': 0.49154597405029093, 'drop_l3': 0.8835693642160674, 'drop_l4': 0.5921375575815719, 'drop_l5': 0.29623137033304137, 'drop_l6': 0.10240777399301759, 'drop_l7': 0.36660860774108656, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:46,757] Trial 489 finished with value: 0.4202041029930115 and parameters: {'drop_l1': 0.18428595524756633, 'drop_l2': 0.40592069483071974, 'drop_l3': 0.8582931884326112, 'drop_l4': 0.6210240384006132, 'drop_l5': 0.27077198404070585, 'drop_l6': 0.10377850066204192, 'drop_l7': 0.33908518726429415, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:48,127] Trial 490 finished with value: 0.5090793371200562 and parameters: {'drop_l1': 0.21734974405723329, 'drop_l2': 0.40836265104069125, 'drop_l3': 0.8881092827581051, 'drop_l4': 0.6033338035720583, 'drop_l5': 0.3137738581540752, 'drop_l6': 0.10352639821124068, 'drop_l7': 0.3613949155017802, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:42:48,181] Trial 491 finished with value: 0.42027583718299866 and parameters: {'drop_l1': 0.19473074791667244, 'drop_l2': 0.2889568958537427, 'drop_l3': 0.8794685114473542, 'drop_l4': 0.6227501706660125, 'drop_l5': 0.2644788142602327, 'drop_l6': 0.16300880845864058, 'drop_l7': 0.3323463001710407, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:43:10,408] Trial 492 finished with value: 0.5233477354049683 and parameters: {'drop_l1': 0.23068101899686425, 'drop_l2': 0.4294169690374052, 'drop_l3': 0.10736052538667347, 'drop_l4': 0.6134145860012238, 'drop_l5': 0.28162625753618525, 'drop_l6': 0.15904412080586608, 'drop_l7': 0.34243400527393036, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:43:28,235] Trial 493 finished with value: 0.5149985551834106 and parameters: {'drop_l1': 0.23497614771095435, 'drop_l2': 0.42758852846024403, 'drop_l3': 0.14235261971982044, 'drop_l4': 0.6112288675567029, 'drop_l5': 0.314867091139556, 'drop_l6': 0.16359861109336943, 'drop_l7': 0.34816341610783985, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:43:29,299] Trial 494 finished with value: 0.5279024839401245 and parameters: {'drop_l1': 0.23286620169501918, 'drop_l2': 0.4299104974009385, 'drop_l3': 0.1402885047817007, 'drop_l4': 0.6104907799466338, 'drop_l5': 0.2967385865566571, 'drop_l6': 0.22488546267808787, 'drop_l7': 0.38094592815736206, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:43:30,373] Trial 495 finished with value: 0.5272441506385803 and parameters: {'drop_l1': 0.24150144904570114, 'drop_l2': 0.4370158168384787, 'drop_l3': 0.15159906954301594, 'drop_l4': 0.6399345841737293, 'drop_l5': 0.30292080301067364, 'drop_l6': 0.205431522729982, 'drop_l7': 0.3841959447686377, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:43:39,773] Trial 496 finished with value: 0.5266655683517456 and parameters: {'drop_l1': 0.24215202558374163, 'drop_l2': 0.3979031385009256, 'drop_l3': 0.14959390632729833, 'drop_l4': 0.644738882115843, 'drop_l5': 0.3029133640939471, 'drop_l6': 0.2198108811934606, 'drop_l7': 0.3140863047319469, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:10,919] Trial 497 finished with value: 0.46044397354125977 and parameters: {'drop_l1': 0.17567323205853907, 'drop_l2': 0.394064397896165, 'drop_l3': 0.8657808578489931, 'drop_l4': 0.643518755464394, 'drop_l5': 0.3650567681215447, 'drop_l6': 0.20405087786117926, 'drop_l7': 0.3967676923114699, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:12,226] Trial 499 finished with value: 0.4201090931892395 and parameters: {'drop_l1': 0.17508942844069197, 'drop_l2': 0.5214282690287663, 'drop_l3': 0.8563192962665734, 'drop_l4': 0.649445736812879, 'drop_l5': 0.2790611871592702, 'drop_l6': 0.11694833857092303, 'drop_l7': 0.3236123429149567, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:12,227] Trial 498 finished with value: 0.5515210628509521 and parameters: {'drop_l1': 0.17867940330941823, 'drop_l2': 0.3970874812161003, 'drop_l3': 0.16397204051774605, 'drop_l4': 0.6382441664213007, 'drop_l5': 0.2746153906538816, 'drop_l6': 0.12030041239673335, 'drop_l7': 0.3164383655285017, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:16,934] Trial 500 finished with value: 0.43671470880508423 and parameters: {'drop_l1': 0.18336514228044204, 'drop_l2': 0.3990239198079139, 'drop_l3': 0.8574432735807352, 'drop_l4': 0.5671638256587274, 'drop_l5': 0.2888709323294204, 'drop_l6': 0.12208790952375645, 'drop_l7': 0.3136539009532134, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:53,995] Trial 501 finished with value: 0.4547320306301117 and parameters: {'drop_l1': 0.20650662497204136, 'drop_l2': 0.3756646281310157, 'drop_l3': 0.5993021293574307, 'drop_l4': 0.5717060845539745, 'drop_l5': 0.28043661719680263, 'drop_l6': 0.12012007450390495, 'drop_l7': 0.3727766817275331, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:55,520] Trial 502 finished with value: 0.4420589804649353 and parameters: {'drop_l1': 0.2178323653640448, 'drop_l2': 0.3728785109925182, 'drop_l3': 0.23700004431425367, 'drop_l4': 0.6595930124423541, 'drop_l5': 0.3305057755111964, 'drop_l6': 0.1419209242910859, 'drop_l7': 0.3283747068138664, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:56,516] Trial 503 finished with value: 0.41203880310058594 and parameters: {'drop_l1': 0.2106506399282784, 'drop_l2': 0.3736533302398328, 'drop_l3': 0.10040483812187895, 'drop_l4': 0.6603678908883089, 'drop_l5': 0.2989144006560427, 'drop_l6': 0.1441488844982249, 'drop_l7': 0.3768475225869836, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:44:57,265] Trial 504 finished with value: 0.41049081087112427 and parameters: {'drop_l1': 0.2112105831449349, 'drop_l2': 0.3404494320994189, 'drop_l3': 0.11332661866592159, 'drop_l4': 0.6616333384540551, 'drop_l5': 0.32884821156460187, 'drop_l6': 0.14262292848193736, 'drop_l7': 0.3734784833347575, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:45:35,565] Trial 505 finished with value: 0.41594186425209045 and parameters: {'drop_l1': 0.25838631459253186, 'drop_l2': 0.3399240604879414, 'drop_l3': 0.26126624728396847, 'drop_l4': 0.6284155203255833, 'drop_l5': 0.3006148257022177, 'drop_l6': 0.14570145996690068, 'drop_l7': 0.39160907304811976, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:45:39,279] Trial 506 finished with value: 0.44509559869766235 and parameters: {'drop_l1': 0.258023627695486, 'drop_l2': 0.33464687760686435, 'drop_l3': 0.1108374794145362, 'drop_l4': 0.629332620893042, 'drop_l5': 0.2537810174176395, 'drop_l6': 0.169578967134802, 'drop_l7': 0.37746173569888825, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:45:39,383] Trial 507 finished with value: 0.43623581528663635 and parameters: {'drop_l1': 0.261859166783419, 'drop_l2': 0.33942619153106857, 'drop_l3': 0.10017617881719051, 'drop_l4': 0.6630439767661671, 'drop_l5': 0.3447703429846659, 'drop_l6': 0.17112723732262103, 'drop_l7': 0.3889949977412176, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:45:39,588] Trial 508 finished with value: 0.4888637065887451 and parameters: {'drop_l1': 0.2577765114987004, 'drop_l2': 0.34475560138840955, 'drop_l3': 0.10493344425143128, 'drop_l4': 0.6643251706072805, 'drop_l5': 0.33869735358131386, 'drop_l6': 0.14649635298729333, 'drop_l7': 0.37982013244817137, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:46:13,598] Trial 509 finished with value: 0.533613920211792 and parameters: {'drop_l1': 0.2614859513673482, 'drop_l2': 0.34956496760131295, 'drop_l3': 0.10518877307024876, 'drop_l4': 0.6622058998565747, 'drop_l5': 0.30428529773983204, 'drop_l6': 0.13948747909162407, 'drop_l7': 0.3819667286126763, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:46:19,682] Trial 511 finished with value: 0.5203703045845032 and parameters: {'drop_l1': 0.15584933849599816, 'drop_l2': 0.35417724176313375, 'drop_l3': 0.6626170621407839, 'drop_l4': 0.6700917316224564, 'drop_l5': 0.3232639737217697, 'drop_l6': 0.1420242112612178, 'drop_l7': 0.38135475553981446, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:46:20,637] Trial 512 finished with value: 0.45319104194641113 and parameters: {'drop_l1': 0.15369416451621754, 'drop_l2': 0.3629698737138105, 'drop_l3': 0.11218640830249363, 'drop_l4': 0.6808065602431719, 'drop_l5': 0.3012707930406722, 'drop_l6': 0.13994530790474666, 'drop_l7': 0.3900030917091623, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:46:20,649] Trial 510 finished with value: 0.42357149720191956 and parameters: {'drop_l1': 0.2711981757468537, 'drop_l2': 0.3497863630029237, 'drop_l3': 0.10087613719054789, 'drop_l4': 0.6675866859028798, 'drop_l5': 0.32562678451378996, 'drop_l6': 0.1401327206632746, 'drop_l7': 0.3728086795403669, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 305 with value: 0.40107059478759766. [I 2021-03-10 18:46:44,044] Trial 513 finished with value: 0.3998320400714874 and parameters: {'drop_l1': 0.15872829858660079, 'drop_l2': 0.124404021283663, 'drop_l3': 0.1191444759931971, 'drop_l4': 0.6775070097550181, 'drop_l5': 0.32337913815894676, 'drop_l6': 0.13903817315530098, 'drop_l7': 0.39426663755264946, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:00,430] Trial 514 finished with value: 0.4319066107273102 and parameters: {'drop_l1': 0.28371345923454006, 'drop_l2': 0.13036445945259686, 'drop_l3': 0.11537045387396193, 'drop_l4': 0.6792950577279449, 'drop_l5': 0.32109134965747116, 'drop_l6': 0.15069989820319227, 'drop_l7': 0.35973652755765706, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:02,772] Trial 515 finished with value: 0.46603626012802124 and parameters: {'drop_l1': 0.21920077609413025, 'drop_l2': 0.12553041006199434, 'drop_l3': 0.12154911814817095, 'drop_l4': 0.6500193530936785, 'drop_l5': 0.3092301054205565, 'drop_l6': 0.1543083537476467, 'drop_l7': 0.8067268597525807, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:02,932] Trial 516 finished with value: 0.4280923008918762 and parameters: {'drop_l1': 0.2828719975337222, 'drop_l2': 0.1232805845697083, 'drop_l3': 0.11926391175664816, 'drop_l4': 0.6561130850820133, 'drop_l5': 0.3043773044415556, 'drop_l6': 0.1561227071159799, 'drop_l7': 0.40532950026297937, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:16,077] Trial 517 finished with value: 0.43937569856643677 and parameters: {'drop_l1': 0.16063973002460347, 'drop_l2': 0.12446816389475683, 'drop_l3': 0.12258569535431571, 'drop_l4': 0.6813585966650274, 'drop_l5': 0.3212905242128948, 'drop_l6': 0.15954565301753454, 'drop_l7': 0.35865412835715926, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:42,292] Trial 518 finished with value: 0.4096141755580902 and parameters: {'drop_l1': 0.1513954887363674, 'drop_l2': 0.12034926849773014, 'drop_l3': 0.13781442681921893, 'drop_l4': 0.7152605707426802, 'drop_l5': 0.3101817746945584, 'drop_l6': 0.17788819480352705, 'drop_l7': 0.4023580416069281, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:44,638] Trial 519 finished with value: 0.44569259881973267 and parameters: {'drop_l1': 0.15755389727469885, 'drop_l2': 0.10393242593186765, 'drop_l3': 0.13937583449763621, 'drop_l4': 0.7109686653468913, 'drop_l5': 0.33998535678304725, 'drop_l6': 0.17558424298644926, 'drop_l7': 0.39600635567819276, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:44,774] Trial 520 finished with value: 0.42641058564186096 and parameters: {'drop_l1': 0.16142479974485271, 'drop_l2': 0.10674693380785302, 'drop_l3': 0.13684251226465982, 'drop_l4': 0.7061237304129534, 'drop_l5': 0.3377109134475075, 'drop_l6': 0.177527793029938, 'drop_l7': 0.40124011797786824, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:47:50,560] Trial 521 finished with value: 0.7550939917564392 and parameters: {'drop_l1': 0.15164731573418458, 'drop_l2': 0.10214768739231747, 'drop_l3': 0.1298929082567931, 'drop_l4': 0.7143380127893768, 'drop_l5': 0.35940245665019505, 'drop_l6': 0.18026745056339832, 'drop_l7': 0.4076651101242585, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:48:24,330] Trial 522 finished with value: 0.45872801542282104 and parameters: {'drop_l1': 0.16335555273576788, 'drop_l2': 0.1076646497844391, 'drop_l3': 0.13887707986945028, 'drop_l4': 0.7258416570638087, 'drop_l5': 0.2902167388768543, 'drop_l6': 0.1747747762894155, 'drop_l7': 0.4044536009328784, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:48:29,430] Trial 523 finished with value: 1.0038810968399048 and parameters: {'drop_l1': 0.19512674144231684, 'drop_l2': 0.16113879163731376, 'drop_l3': 0.1353554726989964, 'drop_l4': 0.726818540755562, 'drop_l5': 0.31122463414198176, 'drop_l6': 0.2032453486446331, 'drop_l7': 0.3899430436248107, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:48:29,552] Trial 524 finished with value: 0.5576558709144592 and parameters: {'drop_l1': 0.1899866513749281, 'drop_l2': 0.16543919717881572, 'drop_l3': 0.13956702617492306, 'drop_l4': 0.7318053002621538, 'drop_l5': 0.29322622584133157, 'drop_l6': 0.19904762578885285, 'drop_l7': 0.41089811759314765, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:48:31,046] Trial 525 finished with value: 0.43158045411109924 and parameters: {'drop_l1': 0.19283374500685477, 'drop_l2': 0.15076002265648714, 'drop_l3': 0.1400881091566985, 'drop_l4': 0.7375470046611747, 'drop_l5': 0.28989071610981776, 'drop_l6': 0.1979882277766853, 'drop_l7': 0.3887719778463935, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:04,264] Trial 526 finished with value: 0.60045325756073 and parameters: {'drop_l1': 0.19120755389603256, 'drop_l2': 0.16880940152972845, 'drop_l3': 0.10284106561342182, 'drop_l4': 0.7370291742073249, 'drop_l5': 0.3031825153687416, 'drop_l6': 0.20265220711547474, 'drop_l7': 0.3723074518237893, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:10,251] Trial 527 finished with value: 0.4960400462150574 and parameters: {'drop_l1': 0.17929702973218842, 'drop_l2': 0.14396801771810555, 'drop_l3': 0.10312171195528387, 'drop_l4': 0.6779409534038304, 'drop_l5': 0.2923292246497886, 'drop_l6': 0.24825207789127118, 'drop_l7': 0.3689312918705473, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:11,390] Trial 528 finished with value: 0.4303101599216461 and parameters: {'drop_l1': 0.1760951384240345, 'drop_l2': 0.13864909199136355, 'drop_l3': 0.16572474526056447, 'drop_l4': 0.7616065595003655, 'drop_l5': 0.26495764023760127, 'drop_l6': 0.1289150600015493, 'drop_l7': 0.3714046279034337, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:11,511] Trial 529 finished with value: 0.43177419900894165 and parameters: {'drop_l1': 0.17759164927929727, 'drop_l2': 0.259918777230258, 'drop_l3': 0.10184859166091827, 'drop_l4': 0.6723824876036615, 'drop_l5': 0.32718056960787695, 'drop_l6': 0.1269773298569159, 'drop_l7': 0.3743968344924742, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:37,558] Trial 530 finished with value: 0.44355693459510803 and parameters: {'drop_l1': 0.170734762724494, 'drop_l2': 0.13309623933491102, 'drop_l3': 0.5730001868840648, 'drop_l4': 0.7551325393288009, 'drop_l5': 0.2620541559959223, 'drop_l6': 0.24002182824211238, 'drop_l7': 0.3687912236715832, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:52,517] Trial 531 finished with value: 0.41125765442848206 and parameters: {'drop_l1': 0.22123172343470632, 'drop_l2': 0.2647046286799136, 'drop_l3': 0.1673608320145159, 'drop_l4': 0.6989757964628521, 'drop_l5': 0.3260334978277932, 'drop_l6': 0.1269688227855042, 'drop_l7': 0.39270358505243336, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:53,772] Trial 533 finished with value: 0.42320355772972107 and parameters: {'drop_l1': 0.21382828955396893, 'drop_l2': 0.11979325301385933, 'drop_l3': 0.12369021352726768, 'drop_l4': 0.6308119158561996, 'drop_l5': 0.3194940832545453, 'drop_l6': 0.15395301044511872, 'drop_l7': 0.41096756125638256, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:49:53,784] Trial 532 finished with value: 0.47494515776634216 and parameters: {'drop_l1': 0.21333743345549627, 'drop_l2': 0.12527025850351353, 'drop_l3': 0.11845816246459852, 'drop_l4': 0.6442302182578492, 'drop_l5': 0.33024756195661115, 'drop_l6': 0.33046940371574396, 'drop_l7': 0.3859229297969086, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:50:08,405] Trial 534 finished with value: 0.4191582202911377 and parameters: {'drop_l1': 0.2233375210208775, 'drop_l2': 0.28581533233063067, 'drop_l3': 0.2607141878498722, 'drop_l4': 0.6373396583561187, 'drop_l5': 0.3201032091223369, 'drop_l6': 0.16415476611919344, 'drop_l7': 0.3921603764143166, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:50:34,523] Trial 535 finished with value: 0.4272640645503998 and parameters: {'drop_l1': 0.20833965700647117, 'drop_l2': 0.2765910459180065, 'drop_l3': 0.1778085680802119, 'drop_l4': 0.7001934677668187, 'drop_l5': 0.3480698262169637, 'drop_l6': 0.11304911424656848, 'drop_l7': 0.4089994190334796, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:50:36,240] Trial 536 finished with value: 0.4274861812591553 and parameters: {'drop_l1': 0.20607311545595142, 'drop_l2': 0.2764105454913486, 'drop_l3': 0.18143596399189021, 'drop_l4': 0.6986405619131648, 'drop_l5': 0.3454666548934409, 'drop_l6': 0.11781761189023686, 'drop_l7': 0.413846008553745, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:50:36,262] Trial 537 finished with value: 0.4367162585258484 and parameters: {'drop_l1': 0.13860422318521773, 'drop_l2': 0.27224120652696937, 'drop_l3': 0.1566632142153188, 'drop_l4': 0.6972670351794781, 'drop_l5': 0.3409982594893454, 'drop_l6': 0.11906884449883653, 'drop_l7': 0.41257893875828333, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:50:41,787] Trial 538 finished with value: 0.4391019344329834 and parameters: {'drop_l1': 0.5665859689446576, 'drop_l2': 0.27137240895480397, 'drop_l3': 0.15931660220741653, 'drop_l4': 0.7032128774651493, 'drop_l5': 0.3560567491222415, 'drop_l6': 0.2285191137474257, 'drop_l7': 0.42097638079247107, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:51:15,742] Trial 539 finished with value: 0.4293108880519867 and parameters: {'drop_l1': 0.14185994049886677, 'drop_l2': 0.26292568453344234, 'drop_l3': 0.15965385032147505, 'drop_l4': 0.699945088245973, 'drop_l5': 0.36042732046418635, 'drop_l6': 0.12204124449623473, 'drop_l7': 0.41382229727362446, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:51:19,085] Trial 540 finished with value: 0.40438979864120483 and parameters: {'drop_l1': 0.14231094190371749, 'drop_l2': 0.24894709603769555, 'drop_l3': 0.1661433084196685, 'drop_l4': 0.6829669640999062, 'drop_l5': 0.3568493066829111, 'drop_l6': 0.13520696588609996, 'drop_l7': 0.3436926983785391, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:51:19,109] Trial 541 finished with value: 0.4542923867702484 and parameters: {'drop_l1': 0.5683168350774886, 'drop_l2': 0.11614144474332663, 'drop_l3': 0.1660843610204388, 'drop_l4': 0.712068953562905, 'drop_l5': 0.3636990663063583, 'drop_l6': 0.2300869106837148, 'drop_l7': 0.35090676358340506, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:51:20,388] Trial 542 finished with value: 0.4234835207462311 and parameters: {'drop_l1': 0.1535124312255851, 'drop_l2': 0.3013001637973549, 'drop_l3': 0.15383709264034584, 'drop_l4': 0.6821098413081959, 'drop_l5': 0.36288629358495667, 'drop_l6': 0.13586559255276692, 'drop_l7': 0.3936494540328551, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:51:55,315] Trial 543 finished with value: 0.4245153069496155 and parameters: {'drop_l1': 0.20067389673352173, 'drop_l2': 0.30017547039429265, 'drop_l3': 0.14828996743225734, 'drop_l4': 0.6858533073460668, 'drop_l5': 0.3736917683636682, 'drop_l6': 0.13419260531329041, 'drop_l7': 0.3987564005404794, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:51:59,982] Trial 545 finished with value: 0.4095475673675537 and parameters: {'drop_l1': 0.12768209746575593, 'drop_l2': 0.2503700472520257, 'drop_l3': 0.1494770872345976, 'drop_l4': 0.7193480187425879, 'drop_l5': 0.37911572501840457, 'drop_l6': 0.10059273148876205, 'drop_l7': 0.3559623039872105, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:52:00,747] Trial 544 finished with value: 0.4672826826572418 and parameters: {'drop_l1': 0.15221205806763888, 'drop_l2': 0.23757736659449047, 'drop_l3': 0.16781348942063454, 'drop_l4': 0.6821323473773218, 'drop_l5': 0.373214932081788, 'drop_l6': 0.6146282720909388, 'drop_l7': 0.3457658505658328, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:52:00,865] Trial 546 finished with value: 0.4253600537776947 and parameters: {'drop_l1': 0.12915933072160737, 'drop_l2': 0.21708806530080182, 'drop_l3': 0.15820082322515963, 'drop_l4': 0.722850793241331, 'drop_l5': 0.38351120143531814, 'drop_l6': 0.16302171341132315, 'drop_l7': 0.3389684791828568, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:52:31,782] Trial 547 finished with value: 0.4795285165309906 and parameters: {'drop_l1': 0.1301257305441031, 'drop_l2': 0.27301279063439265, 'drop_l3': 0.17582000620529253, 'drop_l4': 0.7255759070618624, 'drop_l5': 0.38153633834646905, 'drop_l6': 0.1635281943067057, 'drop_l7': 0.34673438708916277, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:52:41,597] Trial 549 finished with value: 0.4349534511566162 and parameters: {'drop_l1': 0.1430664917808684, 'drop_l2': 0.24918702868747653, 'drop_l3': 0.14697069774029464, 'drop_l4': 0.7149842134377424, 'drop_l5': 0.3828745699815899, 'drop_l6': 0.10013176236068459, 'drop_l7': 0.353805480717233, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:52:41,603] Trial 548 finished with value: 0.4948580265045166 and parameters: {'drop_l1': 0.14579919914170328, 'drop_l2': 0.22713470531574267, 'drop_l3': 0.1695816200673992, 'drop_l4': 0.7253958992817215, 'drop_l5': 0.38050527511012316, 'drop_l6': 0.1923732678473113, 'drop_l7': 0.34170704834038174, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:52:41,605] Trial 550 finished with value: 0.4803401529788971 and parameters: {'drop_l1': 0.16926960197080276, 'drop_l2': 0.288920481564252, 'drop_l3': 0.1784815557105638, 'drop_l4': 0.7474629661347045, 'drop_l5': 0.38590184826019186, 'drop_l6': 0.2131189229189221, 'drop_l7': 0.3554590110021859, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:53:02,207] Trial 551 finished with value: 0.4221927225589752 and parameters: {'drop_l1': 0.16760858954255076, 'drop_l2': 0.23772128185273794, 'drop_l3': 0.18143739222379118, 'drop_l4': 0.7187404941260278, 'drop_l5': 0.39655090446301305, 'drop_l6': 0.10923719386382463, 'drop_l7': 0.3558097614610701, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:53:23,541] Trial 552 finished with value: 0.4136730134487152 and parameters: {'drop_l1': 0.18318593256672683, 'drop_l2': 0.25631972404833003, 'drop_l3': 0.1218812511765157, 'drop_l4': 0.7111970103942633, 'drop_l5': 0.40173236124704953, 'drop_l6': 0.1000011293386335, 'drop_l7': 0.3631091817325459, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:53:23,561] Trial 554 finished with value: 0.43521174788475037 and parameters: {'drop_l1': 0.18597209568847067, 'drop_l2': 0.24720765411442797, 'drop_l3': 0.13054343838177565, 'drop_l4': 0.7144803242955032, 'drop_l5': 0.4023482481430414, 'drop_l6': 0.10544360945584702, 'drop_l7': 0.36314201670641666, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:53:23,562] Trial 553 finished with value: 0.432417094707489 and parameters: {'drop_l1': 0.19215796388826892, 'drop_l2': 0.24360117976417892, 'drop_l3': 0.13264641445178588, 'drop_l4': 0.7167178698406362, 'drop_l5': 0.4035366197506683, 'drop_l6': 0.10029986689242065, 'drop_l7': 0.3650587599708564, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:53:34,499] Trial 555 finished with value: 0.4103584289550781 and parameters: {'drop_l1': 0.1889497017323831, 'drop_l2': 0.2521747194544933, 'drop_l3': 0.14038010514699598, 'drop_l4': 0.7478781524675214, 'drop_l5': 0.40292423874182215, 'drop_l6': 0.10136519784183007, 'drop_l7': 0.37815717065060894, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:05,399] Trial 558 finished with value: 0.4325956702232361 and parameters: {'drop_l1': 0.869825365346584, 'drop_l2': 0.2642052628099395, 'drop_l3': 0.11639927494575417, 'drop_l4': 0.6993928844616081, 'drop_l5': 0.35282771713003286, 'drop_l6': 0.10182472000373716, 'drop_l7': 0.3751738854671368, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:06,628] Trial 556 finished with value: 0.41901201009750366 and parameters: {'drop_l1': 0.18299258390973067, 'drop_l2': 0.23340833099016495, 'drop_l3': 0.11871479907484504, 'drop_l4': 0.6957115485037924, 'drop_l5': 0.3542504478783825, 'drop_l6': 0.10057406702800756, 'drop_l7': 0.37767024788842385, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:06,657] Trial 557 finished with value: 0.4056090712547302 and parameters: {'drop_l1': 0.16314927312096894, 'drop_l2': 0.238357685997018, 'drop_l3': 0.12342202842290503, 'drop_l4': 0.7394345747394697, 'drop_l5': 0.3658659294542297, 'drop_l6': 0.10071380521704011, 'drop_l7': 0.37282573399277286, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:10,567] Trial 559 finished with value: 0.4285224378108978 and parameters: {'drop_l1': 0.17633059085995714, 'drop_l2': 0.24371419952242765, 'drop_l3': 0.1170961085429755, 'drop_l4': 0.7525576540055506, 'drop_l5': 0.3643308179533969, 'drop_l6': 0.10447379465806651, 'drop_l7': 0.3795064731786483, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:46,843] Trial 560 finished with value: 0.4421893060207367 and parameters: {'drop_l1': 0.18013651213205834, 'drop_l2': 0.23160735515677774, 'drop_l3': 0.1421132546010964, 'drop_l4': 0.6953840897851655, 'drop_l5': 0.36207281474916664, 'drop_l6': 0.1021718894071866, 'drop_l7': 0.37922315868095147, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:49,065] Trial 561 finished with value: 0.4551413655281067 and parameters: {'drop_l1': 0.19752703764752977, 'drop_l2': 0.2766020893309047, 'drop_l3': 0.15423876448533114, 'drop_l4': 0.7688810273198602, 'drop_l5': 0.36569758412095604, 'drop_l6': 0.12007943232660534, 'drop_l7': 0.38520796317855427, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:49,111] Trial 562 finished with value: 0.4329114556312561 and parameters: {'drop_l1': 0.16857541717750107, 'drop_l2': 0.21459688165958546, 'drop_l3': 0.15269128670231025, 'drop_l4': 0.7545401152570005, 'drop_l5': 0.3689008653146556, 'drop_l6': 0.11740331199773246, 'drop_l7': 0.3826902068814128, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:54:49,371] Trial 563 finished with value: 0.43802452087402344 and parameters: {'drop_l1': 0.16396026858758433, 'drop_l2': 0.2146570599960526, 'drop_l3': 0.1508731330606044, 'drop_l4': 0.7516203461586445, 'drop_l5': 0.344915408359108, 'drop_l6': 0.11776749997052598, 'drop_l7': 0.393851838842143, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:55:28,646] Trial 564 finished with value: 0.5517333745956421 and parameters: {'drop_l1': 0.16175835563240074, 'drop_l2': 0.22122537635483552, 'drop_l3': 0.1496133139873799, 'drop_l4': 0.754105688139141, 'drop_l5': 0.3685071624276265, 'drop_l6': 0.12765881178296837, 'drop_l7': 0.38944427042040963, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:55:31,214] Trial 565 finished with value: 0.5419484972953796 and parameters: {'drop_l1': 0.16270894511792605, 'drop_l2': 0.26143768793109623, 'drop_l3': 0.1347454576999294, 'drop_l4': 0.6807739650855672, 'drop_l5': 0.3458041989241564, 'drop_l6': 0.13534523133380313, 'drop_l7': 0.39593073889321984, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:55:31,432] Trial 567 finished with value: 0.5352550148963928 and parameters: {'drop_l1': 0.15228595572492898, 'drop_l2': 0.2551455112139684, 'drop_l3': 0.10056372771329328, 'drop_l4': 0.7741190239430478, 'drop_l5': 0.3785522212652549, 'drop_l6': 0.1451985094044845, 'drop_l7': 0.39771329971062047, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:55:31,474] Trial 566 finished with value: 0.46665292978286743 and parameters: {'drop_l1': 0.16310463364675074, 'drop_l2': 0.2490329079983705, 'drop_l3': 0.140000255749224, 'drop_l4': 0.7414265583858315, 'drop_l5': 0.34080136451094234, 'drop_l6': 0.13397399277553343, 'drop_l7': 0.35534162265440755, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:09,569] Trial 568 finished with value: 0.427327960729599 and parameters: {'drop_l1': 0.14645963398533363, 'drop_l2': 0.25350981949698104, 'drop_l3': 0.13243227522750384, 'drop_l4': 0.7465689475160201, 'drop_l5': 0.3391157037515279, 'drop_l6': 0.14917398581630475, 'drop_l7': 0.3601126717828079, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:12,145] Trial 569 finished with value: 0.452737033367157 and parameters: {'drop_l1': 0.20035571916294803, 'drop_l2': 0.2551612564528009, 'drop_l3': 0.12508787751691972, 'drop_l4': 0.737277424364144, 'drop_l5': 0.34149699427569313, 'drop_l6': 0.1499576195594769, 'drop_l7': 0.3679313079042439, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:13,620] Trial 571 finished with value: 0.4406982958316803 and parameters: {'drop_l1': 0.20166135172989538, 'drop_l2': 0.26023172088580776, 'drop_l3': 0.132842898071144, 'drop_l4': 0.7274103047940161, 'drop_l5': 0.3321439644333488, 'drop_l6': 0.15236169140177297, 'drop_l7': 0.36859477438213467, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:13,999] Trial 570 finished with value: 0.4493715167045593 and parameters: {'drop_l1': 0.20139297643616455, 'drop_l2': 0.26481543393343926, 'drop_l3': 0.193472879277224, 'drop_l4': 0.7426453079594293, 'drop_l5': 0.33344699079253054, 'drop_l6': 0.1531921595717156, 'drop_l7': 0.3675946742313774, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:50,240] Trial 572 finished with value: 0.4141158163547516 and parameters: {'drop_l1': 0.2043241257941895, 'drop_l2': 0.288357271225722, 'drop_l3': 0.1672020385336399, 'drop_l4': 0.7240313824382513, 'drop_l5': 0.4002718473763049, 'drop_l6': 0.1744931720785123, 'drop_l7': 0.3690816844835523, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:53,019] Trial 573 finished with value: 0.45171961188316345 and parameters: {'drop_l1': 0.22433383249970745, 'drop_l2': 0.27254004709871177, 'drop_l3': 0.17242966285941105, 'drop_l4': 0.7350046909260214, 'drop_l5': 0.39448581112154424, 'drop_l6': 0.17223190729491514, 'drop_l7': 0.3404681730555212, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:54,524] Trial 574 finished with value: 0.44703859090805054 and parameters: {'drop_l1': 0.228152232566102, 'drop_l2': 0.2915340110829272, 'drop_l3': 0.16767267997203053, 'drop_l4': 0.6725430486029059, 'drop_l5': 0.3948231356060129, 'drop_l6': 0.1750728616508413, 'drop_l7': 0.33582917149495445, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:56:54,536] Trial 575 finished with value: 0.4580000042915344 and parameters: {'drop_l1': 0.23012568943311418, 'drop_l2': 0.22960878791820877, 'drop_l3': 0.16979444143644754, 'drop_l4': 0.6826912777578709, 'drop_l5': 0.38954027353981796, 'drop_l6': 0.17795708490276224, 'drop_l7': 0.34764941106012165, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:57:28,046] Trial 576 finished with value: 0.5719991326332092 and parameters: {'drop_l1': 0.5149607034441931, 'drop_l2': 0.2278384333868969, 'drop_l3': 0.10802393939230978, 'drop_l4': 0.6760385649959185, 'drop_l5': 0.38858676734356623, 'drop_l6': 0.473659119010394, 'drop_l7': 0.33740395470525164, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:57:33,839] Trial 577 finished with value: 0.4531562924385071 and parameters: {'drop_l1': 0.22338265668316826, 'drop_l2': 0.2891337110030023, 'drop_l3': 0.10445011255775473, 'drop_l4': 0.6713532704656766, 'drop_l5': 0.37789590646337534, 'drop_l6': 0.10060835730511618, 'drop_l7': 0.3320366527924002, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:57:33,848] Trial 578 finished with value: 0.46635085344314575 and parameters: {'drop_l1': 0.17794288010461748, 'drop_l2': 0.19124430187651736, 'drop_l3': 0.10044344512952258, 'drop_l4': 0.6601051441260828, 'drop_l5': 0.3162000179296165, 'drop_l6': 0.13204654532776497, 'drop_l7': 0.40613070772831256, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:57:33,857] Trial 579 finished with value: 0.5271916389465332 and parameters: {'drop_l1': 0.12593485823320122, 'drop_l2': 0.18458839553470585, 'drop_l3': 0.10025111002781859, 'drop_l4': 0.6600200037502493, 'drop_l5': 0.35771729874154384, 'drop_l6': 0.1244423703056441, 'drop_l7': 0.4085995561953548, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:57:57,294] Trial 580 finished with value: 0.4131384491920471 and parameters: {'drop_l1': 0.12202706659669307, 'drop_l2': 0.18715858125192067, 'drop_l3': 0.11652841762390827, 'drop_l4': 0.7674083911459779, 'drop_l5': 0.316020482040414, 'drop_l6': 0.12921299205771375, 'drop_l7': 0.40377516435020466, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:14,826] Trial 583 finished with value: 0.4390961527824402 and parameters: {'drop_l1': 0.14948014431869158, 'drop_l2': 0.2404490132983335, 'drop_l3': 0.120776399854524, 'drop_l4': 0.7002794638607874, 'drop_l5': 0.41353084963809794, 'drop_l6': 0.1616402551279529, 'drop_l7': 0.3944385692548583, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:14,840] Trial 581 finished with value: 0.4690666198730469 and parameters: {'drop_l1': 0.1267668551154913, 'drop_l2': 0.23963597644983023, 'drop_l3': 0.12017435206015485, 'drop_l4': 0.7044498759805163, 'drop_l5': 0.3204569648474207, 'drop_l6': 0.18583016788455703, 'drop_l7': 0.3869693164675335, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:14,862] Trial 582 finished with value: 0.4416132867336273 and parameters: {'drop_l1': 0.1512675442071783, 'drop_l2': 0.23957891250496913, 'drop_l3': 0.14688737255993906, 'drop_l4': 0.6953304508042604, 'drop_l5': 0.41162632613379685, 'drop_l6': 0.18800417268914915, 'drop_l7': 0.7132743841023264, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:26,109] Trial 584 finished with value: 0.4321571886539459 and parameters: {'drop_l1': 0.10349147157567705, 'drop_l2': 0.18103063214335505, 'drop_l3': 0.14291559735417367, 'drop_l4': 0.7696897225872322, 'drop_l5': 0.30253942835046305, 'drop_l6': 0.11956476848748562, 'drop_l7': 0.40745550991918184, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:56,811] Trial 587 finished with value: 0.4356674253940582 and parameters: {'drop_l1': 0.11348841999349413, 'drop_l2': 0.2127224399299584, 'drop_l3': 0.1384790140048463, 'drop_l4': 0.7776060124967465, 'drop_l5': 0.30208994772441217, 'drop_l6': 0.11352680242024007, 'drop_l7': 0.42378072814791234, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:56,828] Trial 585 finished with value: 0.42702698707580566 and parameters: {'drop_l1': 0.12387294567296757, 'drop_l2': 0.19475259880986157, 'drop_l3': 0.2097570875081542, 'drop_l4': 0.7560748034048604, 'drop_l5': 0.30401426940769094, 'drop_l6': 0.1195157519449614, 'drop_l7': 0.41409110401817095, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:58:56,871] Trial 586 finished with value: 0.4232083261013031 and parameters: {'drop_l1': 0.10700368803688562, 'drop_l2': 0.18078676209497135, 'drop_l3': 0.10082657892159126, 'drop_l4': 0.7668844812845377, 'drop_l5': 0.28369225701153306, 'drop_l6': 0.12035151748729819, 'drop_l7': 0.4194429933653839, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:59:02,333] Trial 588 finished with value: 0.4744656980037689 and parameters: {'drop_l1': 0.11830501318375873, 'drop_l2': 0.28274184368773364, 'drop_l3': 0.18802361057081401, 'drop_l4': 0.8028757191451491, 'drop_l5': 0.30771039722259885, 'drop_l6': 0.3830870202034005, 'drop_l7': 0.42004354289906554, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:59:39,763] Trial 591 finished with value: 0.48895424604415894 and parameters: {'drop_l1': 0.1360327738575077, 'drop_l2': 0.20351566637019752, 'drop_l3': 0.18763708361998313, 'drop_l4': 0.7944450452035631, 'drop_l5': 0.3264133957497056, 'drop_l6': 0.2929974875002373, 'drop_l7': 0.38157352859229987, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:59:39,768] Trial 590 finished with value: 0.4332851767539978 and parameters: {'drop_l1': 0.14067286899010234, 'drop_l2': 0.20831782656052183, 'drop_l3': 0.19192020130334164, 'drop_l4': 0.7682567234172718, 'drop_l5': 0.32222541732652293, 'drop_l6': 0.13881760923369652, 'drop_l7': 0.3786687600394358, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:59:39,988] Trial 589 finished with value: 0.4361909031867981 and parameters: {'drop_l1': 0.1282747787456089, 'drop_l2': 0.2799075490606992, 'drop_l3': 0.11542158882896658, 'drop_l4': 0.7771705034720164, 'drop_l5': 0.32024814636024806, 'drop_l6': 0.13853026838790958, 'drop_l7': 0.3853747948715399, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 18:59:42,084] Trial 592 finished with value: 0.5059865117073059 and parameters: {'drop_l1': 0.14043503563985646, 'drop_l2': 0.20443939665710847, 'drop_l3': 0.11303802631161805, 'drop_l4': 0.7893350213673671, 'drop_l5': 0.3253607146374611, 'drop_l6': 0.13529760639605537, 'drop_l7': 0.3828493929334127, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:00:20,317] Trial 593 finished with value: 0.49938759207725525 and parameters: {'drop_l1': 0.6547191080807034, 'drop_l2': 0.3061818077117203, 'drop_l3': 0.11698146263609838, 'drop_l4': 0.7346745181569293, 'drop_l5': 0.3726150302612387, 'drop_l6': 0.13809247284416654, 'drop_l7': 0.39717553188099053, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:00:21,259] Trial 595 finished with value: 0.4356479048728943 and parameters: {'drop_l1': 0.17067812104640065, 'drop_l2': 0.26550245493993524, 'drop_l3': 0.6909197373862538, 'drop_l4': 0.5442265000277768, 'drop_l5': 0.350262089001655, 'drop_l6': 0.10248113227643822, 'drop_l7': 0.4030206657898379, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:00:21,829] Trial 596 finished with value: 0.48369675874710083 and parameters: {'drop_l1': 0.17202449447752383, 'drop_l2': 0.25951252387573953, 'drop_l3': 0.12739521523719058, 'drop_l4': 0.7458656320538457, 'drop_l5': 0.3603934679722295, 'drop_l6': 0.15672995275727578, 'drop_l7': 0.3976187254263032, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:00:21,841] Trial 594 finished with value: 0.5157960653305054 and parameters: {'drop_l1': 0.18615129479982923, 'drop_l2': 0.30688558846949554, 'drop_l3': 0.11887116849194163, 'drop_l4': 0.7845717522452869, 'drop_l5': 0.3535081342235451, 'drop_l6': 0.10079434600319513, 'drop_l7': 0.39612555506749864, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:01,637] Trial 597 finished with value: 0.40917959809303284 and parameters: {'drop_l1': 0.1871060132048354, 'drop_l2': 0.2653406085289597, 'drop_l3': 0.15223387660405985, 'drop_l4': 0.7210607148601004, 'drop_l5': 0.3524849247230744, 'drop_l6': 0.1606814337985351, 'drop_l7': 0.3638201038546084, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:02,684] Trial 598 finished with value: 0.435964971780777 and parameters: {'drop_l1': 0.6922095259614055, 'drop_l2': 0.16519685511845353, 'drop_l3': 0.1559875307299219, 'drop_l4': 0.7401572441095878, 'drop_l5': 0.26894459583850827, 'drop_l6': 0.15355464077416833, 'drop_l7': 0.35490978472418117, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:05,242] Trial 600 finished with value: 0.45019054412841797 and parameters: {'drop_l1': 0.15062430663587825, 'drop_l2': 0.1619916784625684, 'drop_l3': 0.14585594432549157, 'drop_l4': 0.7179000362076258, 'drop_l5': 0.27560256724196136, 'drop_l6': 0.16007673421164872, 'drop_l7': 0.36928645651543546, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:05,504] Trial 599 finished with value: 0.4411682188510895 and parameters: {'drop_l1': 0.15204170147892618, 'drop_l2': 0.22443442951043518, 'drop_l3': 0.15335060996882943, 'drop_l4': 0.7146630375800233, 'drop_l5': 0.2774632021058494, 'drop_l6': 0.10085471104032966, 'drop_l7': 0.36049688916057687, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:43,139] Trial 601 finished with value: 0.4542398452758789 and parameters: {'drop_l1': 0.18944502756280573, 'drop_l2': 0.2494488578522139, 'drop_l3': 0.1507383548526007, 'drop_l4': 0.7134453484879817, 'drop_l5': 0.34019119363577416, 'drop_l6': 0.16479156866146155, 'drop_l7': 0.358697619944021, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:44,665] Trial 602 finished with value: 0.4303305745124817 and parameters: {'drop_l1': 0.1918613328446422, 'drop_l2': 0.26992000608354916, 'drop_l3': 0.1578144090821803, 'drop_l4': 0.6505299374467495, 'drop_l5': 0.3491493161067843, 'drop_l6': 0.16742843784128889, 'drop_l7': 0.36055251077331935, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:47,820] Trial 603 finished with value: 0.602432131767273 and parameters: {'drop_l1': 0.18945493396190433, 'drop_l2': 0.2549612467324064, 'drop_l3': 0.16357915433170062, 'drop_l4': 0.6576252252121898, 'drop_l5': 0.34297825821209726, 'drop_l6': 0.12512748286626518, 'drop_l7': 0.35064889556335455, 'activation': 'tanh', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:01:48,106] Trial 604 finished with value: 0.637556791305542 and parameters: {'drop_l1': 0.19349506415877016, 'drop_l2': 0.2724924221912744, 'drop_l3': 0.1771868330299571, 'drop_l4': 0.6497246645161078, 'drop_l5': 0.3413789585462237, 'drop_l6': 0.16537646264446995, 'drop_l7': 0.34082487931058325, 'activation': 'tanh', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:02:24,393] Trial 605 finished with value: 0.7001137137413025 and parameters: {'drop_l1': 0.1878386743099782, 'drop_l2': 0.27239256894706715, 'drop_l3': 0.1655746127733663, 'drop_l4': 0.6522961187776672, 'drop_l5': 0.3744943178495705, 'drop_l6': 0.12654379140761263, 'drop_l7': 0.33545091914990466, 'activation': 'tanh', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:02:27,646] Trial 606 finished with value: 0.47974491119384766 and parameters: {'drop_l1': 0.16169017824159015, 'drop_l2': 0.2816101675803509, 'drop_l3': 0.17555535178328774, 'drop_l4': 0.6870353921406315, 'drop_l5': 0.37539242879888907, 'drop_l6': 0.12933034999599288, 'drop_l7': 0.34780278517980595, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:02:30,814] Trial 607 finished with value: 0.5006290674209595 and parameters: {'drop_l1': 0.17145641905915876, 'drop_l2': 0.28521244107962374, 'drop_l3': 0.1333563534981703, 'drop_l4': 0.6796141044557716, 'drop_l5': 0.37379948563090326, 'drop_l6': 0.7128087441998698, 'drop_l7': 0.33282484136113927, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:02:30,815] Trial 608 finished with value: 0.4395392835140228 and parameters: {'drop_l1': 0.16323317584928354, 'drop_l2': 0.2973912159969737, 'drop_l3': 0.13875971107792368, 'drop_l4': 0.6833589064106071, 'drop_l5': 0.3698384404982948, 'drop_l6': 0.11617877729265594, 'drop_l7': 0.428945805480519, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:03,287] Trial 609 finished with value: 0.41616201400756836 and parameters: {'drop_l1': 0.16369523268459096, 'drop_l2': 0.2903021494521176, 'drop_l3': 0.13416622356834332, 'drop_l4': 0.688755831316872, 'drop_l5': 0.36022619381459353, 'drop_l6': 0.14432889874687832, 'drop_l7': 0.42943592667017183, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:07,948] Trial 610 finished with value: 0.4296944737434387 and parameters: {'drop_l1': 0.17243056935358284, 'drop_l2': 0.2970480835722992, 'drop_l3': 0.1290262513830662, 'drop_l4': 0.49615398917722175, 'drop_l5': 0.3601696942031068, 'drop_l6': 0.1422504973053816, 'drop_l7': 0.4229984550698654, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:11,815] Trial 612 finished with value: 0.4451185166835785 and parameters: {'drop_l1': 0.2089638205260006, 'drop_l2': 0.2296000263431824, 'drop_l3': 0.21704034267900646, 'drop_l4': 0.7599071483359553, 'drop_l5': 0.328299849213598, 'drop_l6': 0.14455960766544734, 'drop_l7': 0.3817243951143824, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:12,080] Trial 611 finished with value: 0.427611768245697 and parameters: {'drop_l1': 0.21085692591827326, 'drop_l2': 0.24497127768367016, 'drop_l3': 0.13329127101143085, 'drop_l4': 0.7017121838856423, 'drop_l5': 0.35358951577404807, 'drop_l6': 0.1462770032022103, 'drop_l7': 0.37654523082881963, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:40,441] Trial 613 finished with value: 0.4721149802207947 and parameters: {'drop_l1': 0.21422263535002517, 'drop_l2': 0.2656439430603836, 'drop_l3': 0.48114064006753887, 'drop_l4': 0.7627679117188122, 'drop_l5': 0.3115470909589517, 'drop_l6': 0.1460517237192646, 'drop_l7': 0.37591152342874823, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:47,587] Trial 614 finished with value: 0.5337688326835632 and parameters: {'drop_l1': 0.10073096626784747, 'drop_l2': 0.2403655486211034, 'drop_l3': 0.20700317971327842, 'drop_l4': 0.6696348167644407, 'drop_l5': 0.31284853836108756, 'drop_l6': 0.15240078684361943, 'drop_l7': 0.3748244653082125, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:51,746] Trial 616 finished with value: 0.5718417763710022 and parameters: {'drop_l1': 0.11810889371600329, 'drop_l2': 0.31309144254456023, 'drop_l3': 0.10435268343068427, 'drop_l4': 0.6665508230176976, 'drop_l5': 0.3114490829924271, 'drop_l6': 0.8037496763901759, 'drop_l7': 0.4067534969095087, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:03:51,764] Trial 615 finished with value: 0.4398573935031891 and parameters: {'drop_l1': 0.11876693577267322, 'drop_l2': 0.25475931412007585, 'drop_l3': 0.10036008860637204, 'drop_l4': 0.7283409833715241, 'drop_l5': 0.31148407074146517, 'drop_l6': 0.18693983967542144, 'drop_l7': 0.37023474106569026, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:04:13,780] Trial 617 finished with value: 0.4494785666465759 and parameters: {'drop_l1': 0.12180210157309385, 'drop_l2': 0.31508598595086545, 'drop_l3': 0.10113196060268317, 'drop_l4': 0.668086543068552, 'drop_l5': 0.38988692921797935, 'drop_l6': 0.18254903155339358, 'drop_l7': 0.4072825422828292, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:04:25,565] Trial 618 finished with value: 0.45635780692100525 and parameters: {'drop_l1': 0.12331085723478044, 'drop_l2': 0.31785710333002076, 'drop_l3': 0.1045169149104433, 'drop_l4': 0.7344726590663689, 'drop_l5': 0.3906678120394193, 'drop_l6': 0.18476759397872722, 'drop_l7': 0.40269466338399174, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:04:29,336] Trial 619 finished with value: 0.4352245330810547 and parameters: {'drop_l1': 0.1438362690995635, 'drop_l2': 0.31033972660239645, 'drop_l3': 0.10139444562774189, 'drop_l4': 0.6458083190600379, 'drop_l5': 0.39035524221164253, 'drop_l6': 0.1136966526241423, 'drop_l7': 0.3998079504740634, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:04:29,339] Trial 620 finished with value: 0.4485059678554535 and parameters: {'drop_l1': 0.13683220777063806, 'drop_l2': 0.22817383064414587, 'drop_l3': 0.151027818606686, 'drop_l4': 0.7069447477332267, 'drop_l5': 0.39332538351514923, 'drop_l6': 0.10195034959125504, 'drop_l7': 0.39860885396371565, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:04:43,386] Trial 621 finished with value: 0.45615553855895996 and parameters: {'drop_l1': 0.14303073478898706, 'drop_l2': 0.2590489505084507, 'drop_l3': 0.4225117351405411, 'drop_l4': 0.7321301155135779, 'drop_l5': 0.41013731757436134, 'drop_l6': 0.11343176546035971, 'drop_l7': 0.434399069465046, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:02,489] Trial 622 finished with value: 0.43048858642578125 and parameters: {'drop_l1': 0.1502929705478077, 'drop_l2': 0.2328063565974965, 'drop_l3': 0.1467105007106128, 'drop_l4': 0.7057438457127093, 'drop_l5': 0.24525210801925248, 'drop_l6': 0.10130127668203764, 'drop_l7': 0.43993130348725534, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:08,070] Trial 623 finished with value: 0.4504387378692627 and parameters: {'drop_l1': 0.1790615086950134, 'drop_l2': 0.2607229095479451, 'drop_l3': 0.123247598834034, 'drop_l4': 0.6928038262127126, 'drop_l5': 0.4072701275468689, 'drop_l6': 0.16281560375704995, 'drop_l7': 0.43718058660163023, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:09,151] Trial 624 finished with value: 0.425983726978302 and parameters: {'drop_l1': 0.10043564247723927, 'drop_l2': 0.2781586372437625, 'drop_l3': 0.12628269860387828, 'drop_l4': 0.6920043568082789, 'drop_l5': 0.3336216852287345, 'drop_l6': 0.16570275160934694, 'drop_l7': 0.43874113621931016, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:16,077] Trial 625 finished with value: 0.45452022552490234 and parameters: {'drop_l1': 0.10137548229418472, 'drop_l2': 0.27387085298019065, 'drop_l3': 0.17788646403707184, 'drop_l4': 0.6979214187955093, 'drop_l5': 0.3375889243193418, 'drop_l6': 0.16881894501933864, 'drop_l7': 0.31990671685952266, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:35,940] Trial 626 finished with value: 0.4813575744628906 and parameters: {'drop_l1': 0.1825960235563381, 'drop_l2': 0.29566603345554243, 'drop_l3': 0.18111850115113906, 'drop_l4': 0.6940845348270961, 'drop_l5': 0.33274798771271646, 'drop_l6': 0.1711473375505703, 'drop_l7': 0.3252865412688328, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:47,802] Trial 627 finished with value: 0.43831658363342285 and parameters: {'drop_l1': 0.1948504938819764, 'drop_l2': 0.29658859284765215, 'drop_l3': 0.19624524806458932, 'drop_l4': 0.6595245954193927, 'drop_l5': 0.3305394494446294, 'drop_l6': 0.12785704767541403, 'drop_l7': 0.3611817641741313, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:48,932] Trial 628 finished with value: 0.4148302972316742 and parameters: {'drop_l1': 0.1997722821412304, 'drop_l2': 0.2714168047892701, 'drop_l3': 0.19582826798475678, 'drop_l4': 0.659154640883638, 'drop_l5': 0.29016217866633714, 'drop_l6': 0.13194209180324498, 'drop_l7': 0.3602637813779955, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:05:52,786] Trial 629 finished with value: 0.5333216190338135 and parameters: {'drop_l1': 0.20042583684866877, 'drop_l2': 0.8010847265063106, 'drop_l3': 0.1631644345296249, 'drop_l4': 0.5156289148323234, 'drop_l5': 0.29507612433864094, 'drop_l6': 0.13022674983782512, 'drop_l7': 0.19214185618382795, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:06:07,645] Trial 630 finished with value: 0.4153803884983063 and parameters: {'drop_l1': 0.19917086570678147, 'drop_l2': 0.29591890349855554, 'drop_l3': 0.1596163854601185, 'drop_l4': 0.661556378169186, 'drop_l5': 0.29007675091125573, 'drop_l6': 0.12939231796998557, 'drop_l7': 0.3560100593766117, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:06:28,708] Trial 631 finished with value: 0.4284651279449463 and parameters: {'drop_l1': 0.20890719818412828, 'drop_l2': 0.2188150219418648, 'drop_l3': 0.16565644825271886, 'drop_l4': 0.5204397745186319, 'drop_l5': 0.41290226431448435, 'drop_l6': 0.13213858930817018, 'drop_l7': 0.3876528944773431, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:06:29,849] Trial 632 finished with value: 0.43635886907577515 and parameters: {'drop_l1': 0.21873657759409026, 'drop_l2': 0.32128863924026696, 'drop_l3': 0.1616526152513418, 'drop_l4': 0.6743866572209982, 'drop_l5': 0.4158814331482914, 'drop_l6': 0.1377423621883294, 'drop_l7': 0.41664539512821763, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:06:32,433] Trial 633 finished with value: 0.4318334460258484 and parameters: {'drop_l1': 0.1575844373463834, 'drop_l2': 0.24600444977002475, 'drop_l3': 0.11897454056583409, 'drop_l4': 0.643980018687606, 'drop_l5': 0.4207040302267612, 'drop_l6': 0.10026368421564984, 'drop_l7': 0.4176288147709418, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:06:39,740] Trial 634 finished with value: 0.4436365067958832 and parameters: {'drop_l1': 0.1644635302019185, 'drop_l2': 0.3232046433021741, 'drop_l3': 0.23782675808374365, 'drop_l4': 0.6387008586859827, 'drop_l5': 0.3762781394310718, 'drop_l6': 0.1003386797868908, 'drop_l7': 0.4190297027631924, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:09,289] Trial 635 finished with value: 0.41989272832870483 and parameters: {'drop_l1': 0.1602013720042989, 'drop_l2': 0.2513213027022555, 'drop_l3': 0.23382687591996013, 'drop_l4': 0.6385908003982816, 'drop_l5': 0.37491002156538533, 'drop_l6': 0.14874309390549018, 'drop_l7': 0.4167086521173751, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:11,302] Trial 636 finished with value: 0.4106198847293854 and parameters: {'drop_l1': 0.17298081273928045, 'drop_l2': 0.19448974198205188, 'drop_l3': 0.23438061352466713, 'drop_l4': 0.6360807024914333, 'drop_l5': 0.3586206651579496, 'drop_l6': 0.1556012988157891, 'drop_l7': 0.4196837748090253, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:12,858] Trial 637 finished with value: 0.48157191276550293 and parameters: {'drop_l1': 0.1736222422278769, 'drop_l2': 0.24824373469107122, 'drop_l3': 0.14014291793405667, 'drop_l4': 0.7471287093558591, 'drop_l5': 0.3764422025158551, 'drop_l6': 0.15574931105532996, 'drop_l7': 0.38377256061809256, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:15,864] Trial 638 finished with value: 0.45984864234924316 and parameters: {'drop_l1': 0.17479126645285142, 'drop_l2': 0.20211792843931303, 'drop_l3': 0.14498955122436377, 'drop_l4': 0.7468696268567003, 'drop_l5': 0.3662788278312371, 'drop_l6': 0.156384809313347, 'drop_l7': 0.8237996416982993, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:48,424] Trial 639 finished with value: 0.44157785177230835 and parameters: {'drop_l1': 0.18061358617276302, 'drop_l2': 0.2880557359327788, 'drop_l3': 0.14524305172360075, 'drop_l4': 0.7509309824284919, 'drop_l5': 0.35465286994342526, 'drop_l6': 0.15681481950942203, 'drop_l7': 0.38468454144785463, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:52,316] Trial 641 finished with value: 0.4415525794029236 and parameters: {'drop_l1': 0.18731974063697523, 'drop_l2': 0.21030298835931097, 'drop_l3': 0.22594051164494983, 'drop_l4': 0.6153236455232279, 'drop_l5': 0.3596727975579685, 'drop_l6': 0.182777584106377, 'drop_l7': 0.4484020938609598, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:52,318] Trial 640 finished with value: 0.41930848360061646 and parameters: {'drop_l1': 0.17770019616471075, 'drop_l2': 0.21387432994746117, 'drop_l3': 0.22413952270783077, 'drop_l4': 0.6176776749110875, 'drop_l5': 0.353036672800149, 'drop_l6': 0.1867312035777891, 'drop_l7': 0.44496927481167514, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:07:53,815] Trial 642 finished with value: 0.4617827534675598 and parameters: {'drop_l1': 0.18259948620135436, 'drop_l2': 0.2828820536081086, 'drop_l3': 0.2351354948739538, 'drop_l4': 0.6151453092828093, 'drop_l5': 0.35263957520843764, 'drop_l6': 0.18598129409112332, 'drop_l7': 0.43926443121893816, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:08:23,454] Trial 643 finished with value: 0.4232896864414215 and parameters: {'drop_l1': 0.18508370809906147, 'drop_l2': 0.22182902267485466, 'drop_l3': 0.22570427556338946, 'drop_l4': 0.6215429981294003, 'drop_l5': 0.3575097239515151, 'drop_l6': 0.1840580243661393, 'drop_l7': 0.44742048703186493, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:08:32,100] Trial 645 finished with value: 0.4488917887210846 and parameters: {'drop_l1': 0.220708375250099, 'drop_l2': 0.2843783758699436, 'drop_l3': 0.24777315440429934, 'drop_l4': 0.6200142863224459, 'drop_l5': 0.4033797802165394, 'drop_l6': 0.19849525557750905, 'drop_l7': 0.34798853975016275, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:08:33,211] Trial 646 finished with value: 0.41040509939193726 and parameters: {'drop_l1': 0.22223743930058273, 'drop_l2': 0.3223553819607709, 'drop_l3': 0.21521598274920212, 'drop_l4': 0.631495852350875, 'drop_l5': 0.39888255345254037, 'drop_l6': 0.20054246471220377, 'drop_l7': 0.42838071833760133, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:08:33,284] Trial 644 finished with value: 0.41299742460250854 and parameters: {'drop_l1': 0.2238435069301841, 'drop_l2': 0.281475620111163, 'drop_l3': 0.21130941933199687, 'drop_l4': 0.6251529365852425, 'drop_l5': 0.38011829868963276, 'drop_l6': 0.17444466892582647, 'drop_l7': 0.35077629857957204, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:08:52,825] Trial 647 finished with value: 0.4314906597137451 and parameters: {'drop_l1': 0.22142872457668333, 'drop_l2': 0.23357831972885174, 'drop_l3': 0.17432204214139296, 'drop_l4': 0.6472265740120933, 'drop_l5': 0.4017398341435128, 'drop_l6': 0.17063368748396793, 'drop_l7': 0.3553437716554907, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:12,764] Trial 648 finished with value: 0.4368062913417816 and parameters: {'drop_l1': 0.21555510334633224, 'drop_l2': 0.26472853536693614, 'drop_l3': 0.257496389238676, 'drop_l4': 0.6499082382764845, 'drop_l5': 0.3813742256091722, 'drop_l6': 0.1708676384828631, 'drop_l7': 0.4268139736679094, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:14,157] Trial 649 finished with value: 0.436411052942276 and parameters: {'drop_l1': 0.20918724137543157, 'drop_l2': 0.31606850412168835, 'drop_l3': 0.20338624549912526, 'drop_l4': 0.5935209932580376, 'drop_l5': 0.39284703344289545, 'drop_l6': 0.21685107400205186, 'drop_l7': 0.42682094922698377, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:14,305] Trial 650 finished with value: 0.4293772578239441 and parameters: {'drop_l1': 0.20098754026369697, 'drop_l2': 0.3203093414399135, 'drop_l3': 0.20853744414111303, 'drop_l4': 0.6321398766188643, 'drop_l5': 0.3886778568801161, 'drop_l6': 0.2043517316400183, 'drop_l7': 0.39159848487174215, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:25,158] Trial 651 finished with value: 0.41901522874832153 and parameters: {'drop_l1': 0.20037547248753346, 'drop_l2': 0.3239558836928609, 'drop_l3': 0.20454573167690646, 'drop_l4': 0.595308635270261, 'drop_l5': 0.3873505621060904, 'drop_l6': 0.21880521416337273, 'drop_l7': 0.42468057023470734, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:55,811] Trial 652 finished with value: 0.456088125705719 and parameters: {'drop_l1': 0.156278979554347, 'drop_l2': 0.32028498832926505, 'drop_l3': 0.20501930785601974, 'drop_l4': 0.5958801995215369, 'drop_l5': 0.36882003054700446, 'drop_l6': 0.2118307153604974, 'drop_l7': 0.39187795724163726, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:58,443] Trial 654 finished with value: 0.45980048179626465 and parameters: {'drop_l1': 0.15677039303990534, 'drop_l2': 0.2989385593004862, 'drop_l3': 0.21503714267580007, 'drop_l4': 0.5907527803991256, 'drop_l5': 0.3651303802399956, 'drop_l6': 0.21306268169701126, 'drop_l7': 0.3700582634658867, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:09:58,454] Trial 653 finished with value: 0.4267067313194275 and parameters: {'drop_l1': 0.15426947995576054, 'drop_l2': 0.30810256078011455, 'drop_l3': 0.21385156103422354, 'drop_l4': 0.5994726886818907, 'drop_l5': 0.36787555145528245, 'drop_l6': 0.2104603799511175, 'drop_l7': 0.371124363071968, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:10:03,634] Trial 655 finished with value: 0.5160428881645203 and parameters: {'drop_l1': 0.15438079388446446, 'drop_l2': 0.303300673392395, 'drop_l3': 0.23513121934573766, 'drop_l4': 0.6327990250381235, 'drop_l5': 0.3645865779042522, 'drop_l6': 0.19952017939967562, 'drop_l7': 0.37647994557387404, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:10:38,585] Trial 656 finished with value: 0.47053536772727966 and parameters: {'drop_l1': 0.1526494374989836, 'drop_l2': 0.2996679883990345, 'drop_l3': 0.22235100434172042, 'drop_l4': 0.629232992885004, 'drop_l5': 0.3470139367088214, 'drop_l6': 0.42318118770193613, 'drop_l7': 0.3780993619134253, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:10:43,421] Trial 658 finished with value: 0.42873144149780273 and parameters: {'drop_l1': 0.13941018262197166, 'drop_l2': 0.33570435095288265, 'drop_l3': 0.24817674406461906, 'drop_l4': 0.6344630707685038, 'drop_l5': 0.3425460294346498, 'drop_l6': 0.11509072334389489, 'drop_l7': 0.3045418090322407, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:10:43,439] Trial 657 finished with value: 0.5110881924629211 and parameters: {'drop_l1': 0.14252366796594668, 'drop_l2': 0.3387536311846099, 'drop_l3': 0.2451897526058878, 'drop_l4': 0.6392257271614225, 'drop_l5': 0.3463268110352751, 'drop_l6': 0.1164388536648335, 'drop_l7': 0.31473185414182125, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:10:44,412] Trial 659 finished with value: 0.4401823878288269 and parameters: {'drop_l1': 0.1381017582490916, 'drop_l2': 0.3609266417633387, 'drop_l3': 0.2546176665567742, 'drop_l4': 0.6743742652497864, 'drop_l5': 0.34496510250211077, 'drop_l6': 0.11819402598632478, 'drop_l7': 0.3160774031266069, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:11:15,281] Trial 660 finished with value: 0.5119823217391968 and parameters: {'drop_l1': 0.17184078919180318, 'drop_l2': 0.34190811825054007, 'drop_l3': 0.24894855548179626, 'drop_l4': 0.6749203700714765, 'drop_l5': 0.425994317000095, 'drop_l6': 0.14878804724691516, 'drop_l7': 0.8523074233194368, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:11:24,007] Trial 663 finished with value: 0.4324234127998352 and parameters: {'drop_l1': 0.17580960235759024, 'drop_l2': 0.3565377393156924, 'drop_l3': 0.1891850629599643, 'drop_l4': 0.7199939472615013, 'drop_l5': 0.40933665928091806, 'drop_l6': 0.14912141121920922, 'drop_l7': 0.4063922322387473, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:11:24,787] Trial 662 finished with value: 0.46843141317367554 and parameters: {'drop_l1': 0.19194298096030635, 'drop_l2': 0.36529857053204867, 'drop_l3': 0.19001453462214976, 'drop_l4': 0.6749393679025945, 'drop_l5': 0.40284155605074723, 'drop_l6': 0.1513750476790512, 'drop_l7': 0.40889074083690746, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:11:24,788] Trial 661 finished with value: 0.44218507409095764 and parameters: {'drop_l1': 0.17022637826427456, 'drop_l2': 0.3581559960248489, 'drop_l3': 0.22917945565010625, 'drop_l4': 0.6747506057860968, 'drop_l5': 0.4003097385339293, 'drop_l6': 0.14994711512919867, 'drop_l7': 0.33466070506961776, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:11:43,501] Trial 664 finished with value: 0.4335343837738037 and parameters: {'drop_l1': 0.1908991854718412, 'drop_l2': 0.36099614185161977, 'drop_l3': 0.18962739516551225, 'drop_l4': 0.7200797801189364, 'drop_l5': 0.4079090126406952, 'drop_l6': 0.15224845896576547, 'drop_l7': 0.4067114502077246, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:06,055] Trial 667 finished with value: 0.42097368836402893 and parameters: {'drop_l1': 0.19220782157064695, 'drop_l2': 0.2722981550728225, 'drop_l3': 0.270836807891014, 'drop_l4': 0.6532684373277172, 'drop_l5': 0.42543041738744575, 'drop_l6': 0.16567003277955689, 'drop_l7': 0.395269121106754, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:06,057] Trial 665 finished with value: 0.43324536085128784 and parameters: {'drop_l1': 0.1980289259721702, 'drop_l2': 0.2644132158663231, 'drop_l3': 0.2719812643819768, 'drop_l4': 0.6517019946100189, 'drop_l5': 0.40400794518006095, 'drop_l6': 0.16010062794998173, 'drop_l7': 0.32898144838164173, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:06,059] Trial 666 finished with value: 0.4606226086616516 and parameters: {'drop_l1': 0.1893121602548221, 'drop_l2': 0.2617005053382534, 'drop_l3': 0.17856147531655922, 'drop_l4': 0.6576274658162297, 'drop_l5': 0.42337917812018705, 'drop_l6': 0.1763121463040655, 'drop_l7': 0.3963756824416063, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:14,464] Trial 668 finished with value: 0.43743228912353516 and parameters: {'drop_l1': 0.21294063594988913, 'drop_l2': 0.26125818720699623, 'drop_l3': 0.2640226112280659, 'drop_l4': 0.6585134325985359, 'drop_l5': 0.42698811208942444, 'drop_l6': 0.1971796293521688, 'drop_l7': 0.39496015710535415, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:47,430] Trial 671 finished with value: 0.4478670656681061 and parameters: {'drop_l1': 0.21235427081532535, 'drop_l2': 0.37888064166261826, 'drop_l3': 0.2667659886665927, 'drop_l4': 0.6097637376293381, 'drop_l5': 0.38500841877448094, 'drop_l6': 0.19571212243539202, 'drop_l7': 0.7881158979624198, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:48,391] Trial 670 finished with value: 0.5095893740653992 and parameters: {'drop_l1': 0.23489516329819593, 'drop_l2': 0.24100869202500594, 'drop_l3': 0.21997380690769758, 'drop_l4': 0.7087187903534546, 'drop_l5': 0.38490696640043576, 'drop_l6': 0.10139964941287395, 'drop_l7': 0.4596668619516883, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:48,394] Trial 669 finished with value: 0.47187432646751404 and parameters: {'drop_l1': 0.21034675012544354, 'drop_l2': 0.24969376264549165, 'drop_l3': 0.2363404168357087, 'drop_l4': 0.7028467558073395, 'drop_l5': 0.38299151020015054, 'drop_l6': 0.1998632549771237, 'drop_l7': 0.6827432080301554, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:12:50,490] Trial 672 finished with value: 0.5105826258659363 and parameters: {'drop_l1': 0.16719243803859463, 'drop_l2': 0.38058608095150154, 'drop_l3': 0.23641928496888945, 'drop_l4': 0.6092737192460962, 'drop_l5': 0.38206560307746296, 'drop_l6': 0.19136386790950588, 'drop_l7': 0.46003937746975776, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:13:29,141] Trial 674 finished with value: 0.44163426756858826 and parameters: {'drop_l1': 0.17335260352338572, 'drop_l2': 0.33013842411459504, 'drop_l3': 0.44758993076399306, 'drop_l4': 0.6868240431347702, 'drop_l5': 0.3265634656847696, 'drop_l6': 0.13275821387580075, 'drop_l7': 0.34573874681384587, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:13:29,205] Trial 673 finished with value: 0.6142688989639282 and parameters: {'drop_l1': 0.23454672796362058, 'drop_l2': 0.23795474315013854, 'drop_l3': 0.22198549279137803, 'drop_l4': 0.7115622073765473, 'drop_l5': 0.38260884859223054, 'drop_l6': 0.13597636570815347, 'drop_l7': 0.45584536870505094, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:13:29,214] Trial 675 finished with value: 0.4971284866333008 and parameters: {'drop_l1': 0.1700479583687113, 'drop_l2': 0.32614054232977974, 'drop_l3': 0.16226757791478683, 'drop_l4': 0.6889409182301772, 'drop_l5': 0.3319294810176137, 'drop_l6': 0.1372975832795591, 'drop_l7': 0.3499741391152241, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:13:29,243] Trial 676 finished with value: 0.45167621970176697 and parameters: {'drop_l1': 0.13793599291976408, 'drop_l2': 0.6645452892263182, 'drop_l3': 0.18234308651382813, 'drop_l4': 0.6353231460943146, 'drop_l5': 0.3380495276697398, 'drop_l6': 0.13659759038059144, 'drop_l7': 0.34667954941493845, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:08,252] Trial 680 finished with value: 0.45053404569625854 and parameters: {'drop_l1': 0.12771476255721745, 'drop_l2': 0.30873152787297425, 'drop_l3': 0.19492716918610592, 'drop_l4': 0.6691828355021122, 'drop_l5': 0.3553123954417623, 'drop_l6': 0.11437356225526785, 'drop_l7': 0.4236466652023123, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:09,081] Trial 677 finished with value: 0.44148167967796326 and parameters: {'drop_l1': 0.13320565400792353, 'drop_l2': 0.28669504277555075, 'drop_l3': 0.19441468427720748, 'drop_l4': 0.6638573438360797, 'drop_l5': 0.35637422266740093, 'drop_l6': 0.11644145985785904, 'drop_l7': 0.6328729823937478, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:09,675] Trial 678 finished with value: 0.45643919706344604 and parameters: {'drop_l1': 0.1290196426425706, 'drop_l2': 0.302379247091803, 'drop_l3': 0.19565720208303017, 'drop_l4': 0.6652430670577301, 'drop_l5': 0.353981160034718, 'drop_l6': 0.11701117796351279, 'drop_l7': 0.638920868571774, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:09,802] Trial 679 finished with value: 0.42761412262916565 and parameters: {'drop_l1': 0.13306590527971893, 'drop_l2': 0.305990069325527, 'drop_l3': 0.14300006556088382, 'drop_l4': 0.5799202492979886, 'drop_l5': 0.3559943573288417, 'drop_l6': 0.1140323594339516, 'drop_l7': 0.4213355642159512, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:48,073] Trial 684 finished with value: 0.8945552110671997 and parameters: {'drop_l1': 0.18329594698415178, 'drop_l2': 0.33523918541856057, 'drop_l3': 0.6297240839987501, 'drop_l4': 0.7280421477975397, 'drop_l5': 0.3996696479156414, 'drop_l6': 0.10029335989385774, 'drop_l7': 0.3703658794612927, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:49,493] Trial 682 finished with value: 0.44374242424964905 and parameters: {'drop_l1': 0.1824847844851309, 'drop_l2': 0.34308086989403824, 'drop_l3': 0.14442183489114221, 'drop_l4': 0.6480681103468423, 'drop_l5': 0.3684996131361467, 'drop_l6': 0.16350414051900425, 'drop_l7': 0.3651612322527009, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:50,508] Trial 681 finished with value: 0.4288005232810974 and parameters: {'drop_l1': 0.20028943710749258, 'drop_l2': 0.2835148459057573, 'drop_l3': 0.1492706427860161, 'drop_l4': 0.6835309259610947, 'drop_l5': 0.3652892421855054, 'drop_l6': 0.23410428219270824, 'drop_l7': 0.36603588909423473, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:14:50,601] Trial 683 finished with value: 0.7943271398544312 and parameters: {'drop_l1': 0.19175135611166869, 'drop_l2': 0.34379139744212833, 'drop_l3': 0.1551765000137483, 'drop_l4': 0.6472476323180962, 'drop_l5': 0.4005761714711499, 'drop_l6': 0.16835060249028336, 'drop_l7': 0.3721657599033824, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:15:28,240] Trial 685 finished with value: 0.4197143018245697 and parameters: {'drop_l1': 0.15755371894458825, 'drop_l2': 0.37251880252643915, 'drop_l3': 0.15647649529079388, 'drop_l4': 0.6450357278393298, 'drop_l5': 0.42749255097494143, 'drop_l6': 0.1621278516692857, 'drop_l7': 0.438870812601455, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:15:30,751] Trial 688 finished with value: 0.5264989137649536 and parameters: {'drop_l1': 0.595390243199597, 'drop_l2': 0.3863457547491631, 'drop_l3': 0.17234392744811453, 'drop_l4': 0.7080432477750729, 'drop_l5': 0.32179431787256374, 'drop_l6': 0.18667711953105376, 'drop_l7': 0.43103165152623835, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:15:30,756] Trial 687 finished with value: 0.5295702219009399 and parameters: {'drop_l1': 0.15464617517419396, 'drop_l2': 0.5740019808141521, 'drop_l3': 0.17047966124252426, 'drop_l4': 0.6247390300242464, 'drop_l5': 0.32525529936580866, 'drop_l6': 0.182339467741461, 'drop_l7': 0.4338588055456578, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:15:30,805] Trial 686 finished with value: 0.4237801134586334 and parameters: {'drop_l1': 0.4996297513790494, 'drop_l2': 0.37833092162975407, 'drop_l3': 0.16921641904298018, 'drop_l4': 0.6869260741861783, 'drop_l5': 0.31974794266173157, 'drop_l6': 0.16857635690915496, 'drop_l7': 0.4383715142344598, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:07,673] Trial 689 finished with value: 0.5021674633026123 and parameters: {'drop_l1': 0.2280902040554636, 'drop_l2': 0.22775467869233212, 'drop_l3': 0.27933296151008946, 'drop_l4': 0.6884494868026191, 'drop_l5': 0.32126611456969734, 'drop_l6': 0.17957815709911, 'drop_l7': 0.4118732706110557, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:12,820] Trial 691 finished with value: 0.5799336433410645 and parameters: {'drop_l1': 0.23211164456188016, 'drop_l2': 0.22367029040040132, 'drop_l3': 0.2829433616957784, 'drop_l4': 0.7280574986900403, 'drop_l5': 0.25891528495996013, 'drop_l6': 0.14268906198331766, 'drop_l7': 0.38828620216392207, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:12,894] Trial 692 finished with value: 0.44339585304260254 and parameters: {'drop_l1': 0.21586279595228403, 'drop_l2': 0.22796640947726504, 'drop_l3': 0.2789050667978401, 'drop_l4': 0.7247289933247794, 'drop_l5': 0.30507865986708654, 'drop_l6': 0.10063443818107366, 'drop_l7': 0.38571955413334047, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:12,927] Trial 690 finished with value: 0.5227962732315063 and parameters: {'drop_l1': 0.22460245636538473, 'drop_l2': 0.24868954873157528, 'drop_l3': 0.24995306171883505, 'drop_l4': 0.7258605222082023, 'drop_l5': 0.30283046788005974, 'drop_l6': 0.14311720263156882, 'drop_l7': 0.41189114387104364, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:41,758] Trial 693 finished with value: 0.4291980266571045 and parameters: {'drop_l1': 0.2150361562638528, 'drop_l2': 0.27411009833040073, 'drop_l3': 0.24694221379723086, 'drop_l4': 0.725985782295179, 'drop_l5': 0.41622746592773996, 'drop_l6': 0.1426279583071913, 'drop_l7': 0.39247529536220077, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:53,495] Trial 694 finished with value: 0.4373297691345215 and parameters: {'drop_l1': 0.20912600246282415, 'drop_l2': 0.27252614080596527, 'drop_l3': 0.1296942499932084, 'drop_l4': 0.6264834395200019, 'drop_l5': 0.41583968052367953, 'drop_l6': 0.2654991048377371, 'drop_l7': 0.33533899139647383, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:55,301] Trial 696 finished with value: 0.4273606240749359 and parameters: {'drop_l1': 0.10103348044880116, 'drop_l2': 0.27074510132137064, 'drop_l3': 0.12807824596549686, 'drop_l4': 0.7020945527483373, 'drop_l5': 0.41026814780364834, 'drop_l6': 0.12564619430907717, 'drop_l7': 0.38924567524617754, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:16:55,315] Trial 695 finished with value: 0.4251073896884918 and parameters: {'drop_l1': 0.11268312466795347, 'drop_l2': 0.2530423755772222, 'drop_l3': 0.1311083834561951, 'drop_l4': 0.6267769404785316, 'drop_l5': 0.41343842547547754, 'drop_l6': 0.2610878909408681, 'drop_l7': 0.32750695700250454, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:17:12,567] Trial 697 finished with value: 0.41531482338905334 and parameters: {'drop_l1': 0.16821795745851117, 'drop_l2': 0.3136944063370289, 'drop_l3': 0.12898141868608948, 'drop_l4': 0.624502220318456, 'drop_l5': 0.4317758026104138, 'drop_l6': 0.12353102558808549, 'drop_l7': 0.4144464271359406, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:17:34,525] Trial 698 finished with value: 0.4089163839817047 and parameters: {'drop_l1': 0.14980811836498617, 'drop_l2': 0.32447264744968773, 'drop_l3': 0.1281708056581422, 'drop_l4': 0.6985446044833141, 'drop_l5': 0.4318827536391323, 'drop_l6': 0.12420925421498903, 'drop_l7': 0.41368268798330265, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:17:36,800] Trial 700 finished with value: 0.4423450827598572 and parameters: {'drop_l1': 0.16749663660147734, 'drop_l2': 0.29178980368623325, 'drop_l3': 0.2105455715095649, 'drop_l4': 0.6712928330635035, 'drop_l5': 0.37484282309069794, 'drop_l6': 0.15869767542051438, 'drop_l7': 0.4154658263628368, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:17:37,076] Trial 699 finished with value: 0.45330291986465454 and parameters: {'drop_l1': 0.16372149395684327, 'drop_l2': 0.5972858406967851, 'drop_l3': 0.2180488267237022, 'drop_l4': 0.6729261889320192, 'drop_l5': 0.43245035140470084, 'drop_l6': 0.15577595455081403, 'drop_l7': 0.4112985571682259, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:17:45,744] Trial 701 finished with value: 0.42796629667282104 and parameters: {'drop_l1': 0.14952170062596595, 'drop_l2': 0.293953259680676, 'drop_l3': 0.25660911571122014, 'drop_l4': 0.6782513154985612, 'drop_l5': 0.3776112856866262, 'drop_l6': 0.15592060515925402, 'drop_l7': 0.3653358159800535, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:18:15,090] Trial 702 finished with value: 0.41945013403892517 and parameters: {'drop_l1': 0.14390792483257975, 'drop_l2': 0.32720594222010646, 'drop_l3': 0.13978657476645906, 'drop_l4': 0.702840730198308, 'drop_l5': 0.37442644288880333, 'drop_l6': 0.10120942394057958, 'drop_l7': 0.41408850506003203, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:18:18,672] Trial 704 finished with value: 0.4200819134712219 and parameters: {'drop_l1': 0.1430536220067951, 'drop_l2': 0.335620800609644, 'drop_l3': 0.11402373748252542, 'drop_l4': 0.7150661766781452, 'drop_l5': 0.3732526449815573, 'drop_l6': 0.10029996288123422, 'drop_l7': 0.4277130607844615, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:18:19,231] Trial 703 finished with value: 0.4452422559261322 and parameters: {'drop_l1': 0.14882698233867192, 'drop_l2': 0.3234088903641119, 'drop_l3': 0.11132039754634596, 'drop_l4': 0.7086141110237258, 'drop_l5': 0.34238935841571533, 'drop_l6': 0.11937151742449133, 'drop_l7': 0.40793395939723964, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:18:23,224] Trial 705 finished with value: 0.437300443649292 and parameters: {'drop_l1': 0.145710823217098, 'drop_l2': 0.32586793308456197, 'drop_l3': 0.11795071540909885, 'drop_l4': 0.7077035148164788, 'drop_l5': 0.2808206896930051, 'drop_l6': 0.10208854012741514, 'drop_l7': 0.4282893948733095, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:18:56,189] Trial 706 finished with value: 0.4216499924659729 and parameters: {'drop_l1': 0.14456177463643238, 'drop_l2': 0.32011865258492156, 'drop_l3': 0.1008293976360012, 'drop_l4': 0.7091750277656697, 'drop_l5': 0.27829424285465887, 'drop_l6': 0.101894922508682, 'drop_l7': 0.4477637956330378, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:01,902] Trial 707 finished with value: 0.42468786239624023 and parameters: {'drop_l1': 0.11662485197284081, 'drop_l2': 0.3176811249404302, 'drop_l3': 0.11559424422527612, 'drop_l4': 0.6968283602776386, 'drop_l5': 0.17775313911148705, 'drop_l6': 0.12524176968948902, 'drop_l7': 0.39590322133030115, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:02,270] Trial 708 finished with value: 0.42793482542037964 and parameters: {'drop_l1': 0.1261355849226258, 'drop_l2': 0.35299462417434996, 'drop_l3': 0.15583097542196778, 'drop_l4': 0.6914570691846034, 'drop_l5': 0.2880511653715282, 'drop_l6': 0.1282769687376893, 'drop_l7': 0.3973974387760527, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:03,091] Trial 709 finished with value: 0.4270494878292084 and parameters: {'drop_l1': 0.11747276771715465, 'drop_l2': 0.35585723384847706, 'drop_l3': 0.1486930239867261, 'drop_l4': 0.7357816985299785, 'drop_l5': 0.333678713661957, 'drop_l6': 0.12941045348802505, 'drop_l7': 0.45220207365535436, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:32,630] Trial 710 finished with value: 0.4195559024810791 and parameters: {'drop_l1': 0.11601805525442431, 'drop_l2': 0.3572751612035509, 'drop_l3': 0.5222446465569636, 'drop_l4': 0.6928435749230709, 'drop_l5': 0.39745868132887263, 'drop_l6': 0.13280432704549638, 'drop_l7': 0.3834059441459484, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:42,884] Trial 711 finished with value: 0.4369681775569916 and parameters: {'drop_l1': 0.1765423955084674, 'drop_l2': 0.3518775336684464, 'drop_l3': 0.1571430989800096, 'drop_l4': 0.6835170047997838, 'drop_l5': 0.39357329423009635, 'drop_l6': 0.13938623376956075, 'drop_l7': 0.3873508929272468, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:44,866] Trial 713 finished with value: 0.42567935585975647 and parameters: {'drop_l1': 0.17683040444572357, 'drop_l2': 0.3062921874215644, 'drop_l3': 0.17825215452667814, 'drop_l4': 0.6909722090850761, 'drop_l5': 0.3951894029784231, 'drop_l6': 0.11632322060373686, 'drop_l7': 0.379242080768409, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:19:44,915] Trial 712 finished with value: 0.44280052185058594 and parameters: {'drop_l1': 0.17617696279569267, 'drop_l2': 0.3518520251078586, 'drop_l3': 0.5307568306885131, 'drop_l4': 0.6875432377381092, 'drop_l5': 0.393635340781774, 'drop_l6': 0.1385550359136963, 'drop_l7': 0.3773410974701214, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:20:01,508] Trial 714 finished with value: 0.41598400473594666 and parameters: {'drop_l1': 0.174426805066348, 'drop_l2': 0.30880852304009865, 'drop_l3': 0.1798428044661094, 'drop_l4': 0.6553938540279605, 'drop_l5': 0.436638837819571, 'drop_l6': 0.10029802011493988, 'drop_l7': 0.3622166057759902, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:20:24,401] Trial 715 finished with value: 0.40256109833717346 and parameters: {'drop_l1': 0.16037040566467087, 'drop_l2': 0.30698463422262895, 'drop_l3': 0.1361625302713624, 'drop_l4': 0.6618035973218298, 'drop_l5': 0.440639372399636, 'drop_l6': 0.11641439152016125, 'drop_l7': 0.3512116955971663, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:20:25,241] Trial 716 finished with value: 0.4301254153251648 and parameters: {'drop_l1': 0.16044883162751777, 'drop_l2': 0.2979516955970939, 'drop_l3': 0.1363826569662339, 'drop_l4': 0.7416296068638176, 'drop_l5': 0.3441310409681427, 'drop_l6': 0.11234457855844006, 'drop_l7': 0.35581835671433737, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:20:26,236] Trial 717 finished with value: 0.43648403882980347 and parameters: {'drop_l1': 0.15943370642669033, 'drop_l2': 0.3758557753313076, 'drop_l3': 0.13453010842959495, 'drop_l4': 0.6102700518608253, 'drop_l5': 0.439019414718688, 'drop_l6': 0.11388434697289289, 'drop_l7': 0.35745971025795664, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:20:32,713] Trial 718 finished with value: 0.4510425627231598 and parameters: {'drop_l1': 0.19097889403717777, 'drop_l2': 0.2919437377809294, 'drop_l3': 0.13178701018680197, 'drop_l4': 0.6071263040338446, 'drop_l5': 0.34682449186668823, 'drop_l6': 0.11634723305672022, 'drop_l7': 0.34817237893056696, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:04,923] Trial 719 finished with value: 0.439217746257782 and parameters: {'drop_l1': 0.13141252023114638, 'drop_l2': 0.2950843210839776, 'drop_l3': 0.13177641668975376, 'drop_l4': 0.7445673246906998, 'drop_l5': 0.3435794006674089, 'drop_l6': 0.114789503486451, 'drop_l7': 0.34568994164458927, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:06,214] Trial 720 finished with value: 0.4427553117275238 and parameters: {'drop_l1': 0.13211177882638162, 'drop_l2': 0.2927692299400278, 'drop_l3': 0.13024790181338397, 'drop_l4': 0.7176262999627188, 'drop_l5': 0.35647692809978304, 'drop_l6': 0.11586298110083411, 'drop_l7': 0.34144640244057367, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:06,340] Trial 721 finished with value: 0.49333009123802185 and parameters: {'drop_l1': 0.13124568357007835, 'drop_l2': 0.29419019846431826, 'drop_l3': 0.12860575102439128, 'drop_l4': 0.716115833788278, 'drop_l5': 0.232051646791125, 'drop_l6': 0.10007468633142318, 'drop_l7': 0.3493822898491366, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:09,413] Trial 722 finished with value: 0.5013414621353149 and parameters: {'drop_l1': 0.13389401719755512, 'drop_l2': 0.5403585995377322, 'drop_l3': 0.1169667433508681, 'drop_l4': 0.6396619007830452, 'drop_l5': 0.30198654249223966, 'drop_l6': 0.10188670633945271, 'drop_l7': 0.3343434630740661, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:45,401] Trial 723 finished with value: 0.4879939556121826 and parameters: {'drop_l1': 0.10004108079262691, 'drop_l2': 0.2806191334370634, 'drop_l3': 0.11299424945837025, 'drop_l4': 0.6423039788148277, 'drop_l5': 0.3598874211499854, 'drop_l6': 0.142331105529501, 'drop_l7': 0.32447365185568683, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:47,107] Trial 724 finished with value: 0.4894779324531555 and parameters: {'drop_l1': 0.10142970998180291, 'drop_l2': 0.3157906265681349, 'drop_l3': 0.10146411348514402, 'drop_l4': 0.6399837954307483, 'drop_l5': 0.3017511854727245, 'drop_l6': 0.13968659899892105, 'drop_l7': 0.3214793202391857, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:47,241] Trial 725 finished with value: 0.6196959018707275 and parameters: {'drop_l1': 0.15270466227611396, 'drop_l2': 0.3172893172807464, 'drop_l3': 0.11741222132444037, 'drop_l4': 0.639279410650073, 'drop_l5': 0.8083027017963013, 'drop_l6': 0.14485544049914043, 'drop_l7': 0.3139987866006598, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:21:48,492] Trial 726 finished with value: 0.5911737680435181 and parameters: {'drop_l1': 0.54548013472757, 'drop_l2': 0.31570081359578345, 'drop_l3': 0.10225234354338958, 'drop_l4': 0.6453855166023555, 'drop_l5': 0.32928823061269535, 'drop_l6': 0.14067163823508652, 'drop_l7': 0.30292259275728406, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:22:25,462] Trial 727 finished with value: 0.4373458921909332 and parameters: {'drop_l1': 0.14330652178742107, 'drop_l2': 0.25991114989696273, 'drop_l3': 0.1014580444740166, 'drop_l4': 0.5760411531490975, 'drop_l5': 0.31315312561949793, 'drop_l6': 0.14034182358716696, 'drop_l7': 0.3683399801063459, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:22:27,916] Trial 729 finished with value: 0.4427250325679779 and parameters: {'drop_l1': 0.15708002512729305, 'drop_l2': 0.2610708169652623, 'drop_l3': 0.10008220088434946, 'drop_l4': 0.5761347428649966, 'drop_l5': 0.33024609373389047, 'drop_l6': 0.1283408482973124, 'drop_l7': 0.45548850134507773, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:22:28,009] Trial 728 finished with value: 0.5157855153083801 and parameters: {'drop_l1': 0.1537432392205333, 'drop_l2': 0.3094370940021786, 'drop_l3': 0.1440074084140532, 'drop_l4': 0.6654523491197267, 'drop_l5': 0.41455572387821343, 'drop_l6': 0.1294155279265475, 'drop_l7': 0.36688174017029523, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:22:28,038] Trial 730 finished with value: 0.5012813210487366 and parameters: {'drop_l1': 0.15371514749311013, 'drop_l2': 0.2599953382370413, 'drop_l3': 0.14977689982504025, 'drop_l4': 0.6696088186852598, 'drop_l5': 0.42095026637349164, 'drop_l6': 0.10008515825085382, 'drop_l7': 0.3653202986532151, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:05,489] Trial 731 finished with value: 0.48893946409225464 and parameters: {'drop_l1': 0.16224835113511998, 'drop_l2': 0.33605744108051516, 'drop_l3': 0.14875447799919456, 'drop_l4': 0.6662657323976408, 'drop_l5': 0.44923040932322506, 'drop_l6': 0.12519124168965595, 'drop_l7': 0.45911491902042356, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:10,140] Trial 733 finished with value: 0.4354749619960785 and parameters: {'drop_l1': 0.16106955926531996, 'drop_l2': 0.29906858026082106, 'drop_l3': 0.16428334038391157, 'drop_l4': 0.682455804082662, 'drop_l5': 0.3797265577506607, 'drop_l6': 0.1520389226869532, 'drop_l7': 0.4632016308016885, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:10,205] Trial 732 finished with value: 0.4555296003818512 and parameters: {'drop_l1': 0.16530063846147489, 'drop_l2': 0.3348636745657503, 'drop_l3': 0.1466908459508354, 'drop_l4': 0.667559006425308, 'drop_l5': 0.37296288547907835, 'drop_l6': 0.15175634417510925, 'drop_l7': 0.43687680172107907, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:10,262] Trial 734 finished with value: 0.423806756734848 and parameters: {'drop_l1': 0.1656805311093017, 'drop_l2': 0.2796946193139423, 'drop_l3': 0.16598834090321443, 'drop_l4': 0.6806322479066175, 'drop_l5': 0.45157047487924307, 'drop_l6': 0.1539668772701459, 'drop_l7': 0.29383941353653503, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:40,612] Trial 735 finished with value: 0.43833351135253906 and parameters: {'drop_l1': 0.19504911372053121, 'drop_l2': 0.27935802944092064, 'drop_l3': 0.20377624694917526, 'drop_l4': 0.6824949873055283, 'drop_l5': 0.37033986700809746, 'drop_l6': 0.15353986120492852, 'drop_l7': 0.4311138849372739, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:52,354] Trial 737 finished with value: 0.48551708459854126 and parameters: {'drop_l1': 0.11659178554420782, 'drop_l2': 0.23745306191372073, 'drop_l3': 0.1963114815153281, 'drop_l4': 0.5477336097902046, 'drop_l5': 0.3879462574587952, 'drop_l6': 0.44444560828567564, 'drop_l7': 0.3337228162929605, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:53,378] Trial 738 finished with value: 0.4655846059322357 and parameters: {'drop_l1': 0.19754576285453332, 'drop_l2': 0.28153378690449443, 'drop_l3': 0.20882278563015053, 'drop_l4': 0.7045900585067892, 'drop_l5': 0.3646767506955439, 'drop_l6': 0.4503157443555592, 'drop_l7': 0.3533009702049237, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:23:53,506] Trial 736 finished with value: 0.8462347984313965 and parameters: {'drop_l1': 0.6214053469466301, 'drop_l2': 0.24059492602667745, 'drop_l3': 0.23191047107967402, 'drop_l4': 0.7019387465496973, 'drop_l5': 0.3974565847113241, 'drop_l6': 0.11669317060191403, 'drop_l7': 0.3356069126996421, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:24:12,745] Trial 739 finished with value: 0.8208076357841492 and parameters: {'drop_l1': 0.12067366147408456, 'drop_l2': 0.23861878026722916, 'drop_l3': 0.2287709032532675, 'drop_l4': 0.7061622630916737, 'drop_l5': 0.3957167356558258, 'drop_l6': 0.11440547459599201, 'drop_l7': 0.3470295914117574, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:24:34,722] Trial 740 finished with value: 0.5216800570487976 and parameters: {'drop_l1': 0.1968481690209515, 'drop_l2': 0.19681710379764392, 'drop_l3': 0.12450225517546482, 'drop_l4': 0.7279760636501561, 'drop_l5': 0.3971673509418437, 'drop_l6': 0.10042191184689006, 'drop_l7': 0.3796892492837838, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:24:36,865] Trial 741 finished with value: 0.5807065963745117 and parameters: {'drop_l1': 0.4069899404353629, 'drop_l2': 0.2463325053559173, 'drop_l3': 0.23531580684875353, 'drop_l4': 0.7336599846184604, 'drop_l5': 0.42861566026015, 'drop_l6': 0.8249950935909687, 'drop_l7': 0.3792541218149998, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:24:36,965] Trial 742 finished with value: 0.4146191477775574 and parameters: {'drop_l1': 0.14011896863819057, 'drop_l2': 0.33276344385061357, 'drop_l3': 0.18260570678506582, 'drop_l4': 0.7237283206512415, 'drop_l5': 0.4322099271914257, 'drop_l6': 0.1296731004353767, 'drop_l7': 0.3788269189347817, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:24:46,206] Trial 743 finished with value: 0.4299767017364502 and parameters: {'drop_l1': 0.14314467570726336, 'drop_l2': 0.3300819717379803, 'drop_l3': 0.12387049489662816, 'drop_l4': 0.7409105467461662, 'drop_l5': 0.42349942646979416, 'drop_l6': 0.10105590248099633, 'drop_l7': 0.38189127276185153, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:25:15,225] Trial 744 finished with value: 0.6593260169029236 and parameters: {'drop_l1': 0.18001286789462234, 'drop_l2': 0.3246501290239985, 'drop_l3': 0.4620338707761699, 'drop_l4': 0.7370346255001279, 'drop_l5': 0.8934726194889395, 'drop_l6': 0.12937569845507968, 'drop_l7': 0.3955473267401881, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:25:18,963] Trial 745 finished with value: 0.6055864095687866 and parameters: {'drop_l1': 0.13752101529051688, 'drop_l2': 0.3275081268614307, 'drop_l3': 0.10042934195138598, 'drop_l4': 0.652889704338285, 'drop_l5': 0.8950533591802761, 'drop_l6': 0.16489739015530047, 'drop_l7': 0.399286528729417, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:25:19,530] Trial 746 finished with value: 0.5285215973854065 and parameters: {'drop_l1': 0.7639512029232782, 'drop_l2': 0.32562571130615037, 'drop_l3': 0.14135183951093783, 'drop_l4': 0.6562979115604836, 'drop_l5': 0.4123016022515528, 'drop_l6': 0.1672387564896883, 'drop_l7': 0.40221753795926435, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:25:22,503] Trial 747 finished with value: 0.5352977514266968 and parameters: {'drop_l1': 0.7685639762487717, 'drop_l2': 0.3081190533214955, 'drop_l3': 0.1387289250613551, 'drop_l4': 0.656889745624578, 'drop_l5': 0.33115947221966974, 'drop_l6': 0.16746807440458944, 'drop_l7': 0.40195898975098054, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:25:54,363] Trial 748 finished with value: 0.42248621582984924 and parameters: {'drop_l1': 0.10129632957114998, 'drop_l2': 0.30813869010448114, 'drop_l3': 0.4948433978093002, 'drop_l4': 0.6570874552541021, 'drop_l5': 0.3332952492237207, 'drop_l6': 0.16609144539222506, 'drop_l7': 0.42305329797721986, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:00,867] Trial 750 finished with value: 0.48865264654159546 and parameters: {'drop_l1': 0.17727079060416287, 'drop_l2': 0.30531126715989426, 'drop_l3': 0.16478109096305743, 'drop_l4': 0.6163249714471078, 'drop_l5': 0.3357956103715579, 'drop_l6': 0.10070140472258249, 'drop_l7': 0.4268797317828372, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:01,561] Trial 751 finished with value: 0.4285790026187897 and parameters: {'drop_l1': 0.17608860308258664, 'drop_l2': 0.2753384894532521, 'drop_l3': 0.8999505037533102, 'drop_l4': 0.6149256249481866, 'drop_l5': 0.3109512950168608, 'drop_l6': 0.138103508360574, 'drop_l7': 0.42421824326330065, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:01,563] Trial 749 finished with value: 0.48863348364830017 and parameters: {'drop_l1': 0.10020902699211778, 'drop_l2': 0.31058096293396437, 'drop_l3': 0.17050110913751196, 'drop_l4': 0.658367353911636, 'drop_l5': 0.33199201917737703, 'drop_l6': 0.1699914428695422, 'drop_l7': 0.4220968863238733, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:23,964] Trial 752 finished with value: 0.443448543548584 and parameters: {'drop_l1': 0.18573762509217026, 'drop_l2': 0.2677920615992393, 'drop_l3': 0.16643928312652323, 'drop_l4': 0.6039620379528704, 'drop_l5': 0.3148563259096559, 'drop_l6': 0.14334006288149692, 'drop_l7': 0.43735359840161203, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:42,339] Trial 753 finished with value: 0.43336206674575806 and parameters: {'drop_l1': 0.1848819855328998, 'drop_l2': 0.27443469642030954, 'drop_l3': 0.18084917325259645, 'drop_l4': 0.697502007185126, 'drop_l5': 0.3130824823198218, 'drop_l6': 0.14089330441069342, 'drop_l7': 0.44425745465176336, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:43,161] Trial 754 finished with value: 0.44800248742103577 and parameters: {'drop_l1': 0.18929506145218394, 'drop_l2': 0.26583596403681303, 'drop_l3': 0.1191629350223374, 'drop_l4': 0.6960256065162865, 'drop_l5': 0.355521141894561, 'drop_l6': 0.1240090997687258, 'drop_l7': 0.4407744455017159, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:43,162] Trial 755 finished with value: 0.44219574332237244 and parameters: {'drop_l1': 0.18464894024069417, 'drop_l2': 0.2122724016438271, 'drop_l3': 0.12326750343707817, 'drop_l4': 0.6894542124756572, 'drop_l5': 0.3553712487095462, 'drop_l6': 0.12352616008816804, 'drop_l7': 0.46867230022614803, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:26:54,170] Trial 756 finished with value: 0.4776710867881775 and parameters: {'drop_l1': 0.2000232867584385, 'drop_l2': 0.3456742119651476, 'drop_l3': 0.11196861144679768, 'drop_l4': 0.7004590816101095, 'drop_l5': 0.3631326568778626, 'drop_l6': 0.40860266380265375, 'drop_l7': 0.44556173885376316, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:27:21,631] Trial 757 finished with value: 0.4554092288017273 and parameters: {'drop_l1': 0.20217965848361807, 'drop_l2': 0.34126922570826124, 'drop_l3': 0.11578587855809461, 'drop_l4': 0.6840771705314409, 'drop_l5': 0.3522923090818908, 'drop_l6': 0.4088211403372141, 'drop_l7': 0.47093164865073917, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:27:23,364] Trial 759 finished with value: 0.41577544808387756 and parameters: {'drop_l1': 0.20669563979066963, 'drop_l2': 0.3405993256501797, 'drop_l3': 0.2202409185514365, 'drop_l4': 0.6316257587699986, 'drop_l5': 0.27051120453914596, 'drop_l6': 0.1525203031069735, 'drop_l7': 0.35815076366381177, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:27:23,392] Trial 758 finished with value: 0.42566150426864624 and parameters: {'drop_l1': 0.12330695919614211, 'drop_l2': 0.2491033956850695, 'drop_l3': 0.613233796034687, 'drop_l4': 0.631311831587764, 'drop_l5': 0.26501860249646847, 'drop_l6': 0.11633451532988069, 'drop_l7': 0.3633303576664639, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:27:28,020] Trial 760 finished with value: 0.4241393208503723 and parameters: {'drop_l1': 0.12652518216609376, 'drop_l2': 0.3931031469870347, 'drop_l3': 0.15265989383108186, 'drop_l4': 0.7164310869221103, 'drop_l5': 0.2694839130561093, 'drop_l6': 0.11331138826447015, 'drop_l7': 0.3597763145427184, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:01,917] Trial 761 finished with value: 0.44774356484413147 and parameters: {'drop_l1': 0.12939149778526166, 'drop_l2': 0.41512799655048266, 'drop_l3': 0.1503264319711005, 'drop_l4': 0.7574286191635479, 'drop_l5': 0.2649039919751643, 'drop_l6': 0.1509336848180526, 'drop_l7': 0.3581103753146641, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:05,765] Trial 762 finished with value: 0.4280862808227539 and parameters: {'drop_l1': 0.12600145486729558, 'drop_l2': 0.2886052061755558, 'drop_l3': 0.14666521490537815, 'drop_l4': 0.67468783328328, 'drop_l5': 0.29300659922360056, 'drop_l6': 0.11255305800820609, 'drop_l7': 0.3327672457221181, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:05,945] Trial 763 finished with value: 0.4173014760017395 and parameters: {'drop_l1': 0.14801688766489735, 'drop_l2': 0.389272750346149, 'drop_l3': 0.37915966893923136, 'drop_l4': 0.7211798295549531, 'drop_l5': 0.2924907834459316, 'drop_l6': 0.11302397048587395, 'drop_l7': 0.30754160561848704, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:07,074] Trial 764 finished with value: 0.4584415555000305 and parameters: {'drop_l1': 0.15116242567780672, 'drop_l2': 0.290116322003818, 'drop_l3': 0.13648067519931742, 'drop_l4': 0.6753444883132809, 'drop_l5': 0.45019098335140273, 'drop_l6': 0.15205630629419373, 'drop_l7': 0.3277866321947061, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:39,941] Trial 765 finished with value: 0.5603342652320862 and parameters: {'drop_l1': 0.1513097526127106, 'drop_l2': 0.2954090168118989, 'drop_l3': 0.24402447074004427, 'drop_l4': 0.6695975173994189, 'drop_l5': 0.29344091624724467, 'drop_l6': 0.6339127421942584, 'drop_l7': 0.3139135940682923, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:47,532] Trial 768 finished with value: 0.5585507750511169 and parameters: {'drop_l1': 0.16271447337784337, 'drop_l2': 0.26028165918231294, 'drop_l3': 0.24976203365236638, 'drop_l4': 0.7140182074503323, 'drop_l5': 0.38459085432699386, 'drop_l6': 0.10093658607138944, 'drop_l7': 0.3897946077116228, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:47,534] Trial 766 finished with value: 0.45747122168540955 and parameters: {'drop_l1': 0.1493365241498663, 'drop_l2': 0.37289343173094974, 'drop_l3': 0.25917924992989355, 'drop_l4': 0.7146618718747277, 'drop_l5': 0.3833110194197641, 'drop_l6': 0.3498742741461444, 'drop_l7': 0.4081767201579568, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:28:47,535] Trial 767 finished with value: 0.5271490812301636 and parameters: {'drop_l1': 0.1601296823800347, 'drop_l2': 0.22182182553572158, 'drop_l3': 0.1349944048726449, 'drop_l4': 0.47767324956759083, 'drop_l5': 0.4508636428848995, 'drop_l6': 0.10029121237080904, 'drop_l7': 0.3936214585281294, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:29:10,896] Trial 769 finished with value: 0.49156951904296875 and parameters: {'drop_l1': 0.1662272759301077, 'drop_l2': 0.3671702300631596, 'drop_l3': 0.2583654748230067, 'drop_l4': 0.6474990416606012, 'drop_l5': 0.3859273575200602, 'drop_l6': 0.1335916748254558, 'drop_l7': 0.41012176072210366, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:29:30,420] Trial 770 finished with value: 0.4725084900856018 and parameters: {'drop_l1': 0.11052076864255617, 'drop_l2': 0.3672100718966998, 'drop_l3': 0.19134640401743824, 'drop_l4': 0.6444382293073522, 'drop_l5': 0.4052452613013502, 'drop_l6': 0.13273775782532835, 'drop_l7': 0.4168810732751871, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:29:30,468] Trial 772 finished with value: 0.4092589020729065 and parameters: {'drop_l1': 0.11209340138458183, 'drop_l2': 0.3560242446678141, 'drop_l3': 0.19056722996507638, 'drop_l4': 0.6494595298869803, 'drop_l5': 0.41115812807685936, 'drop_l6': 0.13350642799854687, 'drop_l7': 0.4136017598545769, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:29:30,482] Trial 771 finished with value: 0.43200474977493286 and parameters: {'drop_l1': 0.8096338504555047, 'drop_l2': 0.47123228561246455, 'drop_l3': 0.30357095619218455, 'drop_l4': 0.6427479741701603, 'drop_l5': 0.4097889653025782, 'drop_l6': 0.13192422378013102, 'drop_l7': 0.288485384172163, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:29:42,553] Trial 773 finished with value: 0.4808827042579651 and parameters: {'drop_l1': 0.11272896814514134, 'drop_l2': 0.34567716047479596, 'drop_l3': 0.19781043625529585, 'drop_l4': 0.623747945232429, 'drop_l5': 0.41267359137077453, 'drop_l6': 0.1823489343575363, 'drop_l7': 0.3706746223190733, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:13,234] Trial 774 finished with value: 0.47087258100509644 and parameters: {'drop_l1': 0.20810695707196097, 'drop_l2': 0.20147353517545188, 'drop_l3': 0.1954183922699148, 'drop_l4': 0.624384633474191, 'drop_l5': 0.41874832733000256, 'drop_l6': 0.17325811584171105, 'drop_l7': 0.45103945623604236, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:14,136] Trial 775 finished with value: 0.42636290192604065 and parameters: {'drop_l1': 0.10995689497947525, 'drop_l2': 0.3451383706326951, 'drop_l3': 0.201637159193867, 'drop_l4': 0.6336456010811721, 'drop_l5': 0.4341129035624034, 'drop_l6': 0.17946210972932078, 'drop_l7': 0.45018256462156997, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:14,774] Trial 776 finished with value: 0.4959884285926819 and parameters: {'drop_l1': 0.12474508992801189, 'drop_l2': 0.34760006505330154, 'drop_l3': 0.1911685703791128, 'drop_l4': 0.623106353741193, 'drop_l5': 0.4321608535843691, 'drop_l6': 0.1797793736867811, 'drop_l7': 0.4517302651427641, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:19,546] Trial 777 finished with value: 0.4455813765525818 and parameters: {'drop_l1': 0.11121964473768828, 'drop_l2': 0.8608126501842366, 'drop_l3': 0.18826774621402417, 'drop_l4': 0.6026325691410475, 'drop_l5': 0.4287989578095912, 'drop_l6': 0.12667168901328324, 'drop_l7': 0.4512800888771359, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:57,025] Trial 778 finished with value: 0.44244998693466187 and parameters: {'drop_l1': 0.10735814639467602, 'drop_l2': 0.3174876605946823, 'drop_l3': 0.20637387226343945, 'drop_l4': 0.6062268000248195, 'drop_l5': 0.4347047416437503, 'drop_l6': 0.12492749425942823, 'drop_l7': 0.4331173984229366, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:58,843] Trial 779 finished with value: 0.42117661237716675 and parameters: {'drop_l1': 0.13418203442253213, 'drop_l2': 0.31768522738917004, 'drop_l3': 0.18403749527216462, 'drop_l4': 0.7494456737250418, 'drop_l5': 0.43694172801302184, 'drop_l6': 0.10066079774499903, 'drop_l7': 0.4274102729116077, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:30:59,380] Trial 780 finished with value: 0.43298792839050293 and parameters: {'drop_l1': 0.10692191848980899, 'drop_l2': 0.31838262844480053, 'drop_l3': 0.18390959915427196, 'drop_l4': 0.6034850895430373, 'drop_l5': 0.4064401763742933, 'drop_l6': 0.10007978900423654, 'drop_l7': 0.4323681900315459, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:31:01,833] Trial 781 finished with value: 0.4405056834220886 and parameters: {'drop_l1': 0.13443645057260317, 'drop_l2': 0.3108422639064843, 'drop_l3': 0.17663349775364526, 'drop_l4': 0.7545774808501825, 'drop_l5': 0.3991120959072265, 'drop_l6': 0.15427580714005817, 'drop_l7': 0.42529144780495637, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:31:39,297] Trial 782 finished with value: 0.44293326139450073 and parameters: {'drop_l1': 0.1009984834339049, 'drop_l2': 0.25202997391117804, 'drop_l3': 0.2173797064781551, 'drop_l4': 0.6920066995047162, 'drop_l5': 0.4088767606241946, 'drop_l6': 0.15690236261283352, 'drop_l7': 0.4134581470865581, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:31:42,395] Trial 784 finished with value: 0.4187646508216858 and parameters: {'drop_l1': 0.10068733153420269, 'drop_l2': 0.28031518485191054, 'drop_l3': 0.21970957224130713, 'drop_l4': 0.6871317611423702, 'drop_l5': 0.3979838729440539, 'drop_l6': 0.15448262176623248, 'drop_l7': 0.40300012654182454, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:31:42,456] Trial 783 finished with value: 0.43775081634521484 and parameters: {'drop_l1': 0.13637362476441653, 'drop_l2': 0.25198319212889014, 'drop_l3': 0.1735525704105118, 'drop_l4': 0.6985547443639402, 'drop_l5': 0.4023626352109536, 'drop_l6': 0.15798701298918685, 'drop_l7': 0.4158224058057881, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:31:42,901] Trial 785 finished with value: 0.4191771149635315 and parameters: {'drop_l1': 0.14130925906662647, 'drop_l2': 0.2540086866038233, 'drop_l3': 0.16270136712655928, 'drop_l4': 0.6905918793671351, 'drop_l5': 0.3767076039781203, 'drop_l6': 0.14179672652735142, 'drop_l7': 0.41125720252400594, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:32:18,720] Trial 786 finished with value: 0.4329988956451416 and parameters: {'drop_l1': 0.14297609672017406, 'drop_l2': 0.2833523138149324, 'drop_l3': 0.1652019322609389, 'drop_l4': 0.6772526784527071, 'drop_l5': 0.37270153745447404, 'drop_l6': 0.1401855258577243, 'drop_l7': 0.40396174515468936, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:32:22,317] Trial 788 finished with value: 0.5168795585632324 and parameters: {'drop_l1': 0.1466934129358165, 'drop_l2': 0.3006751927349787, 'drop_l3': 0.16897150051885637, 'drop_l4': 0.6741931341362617, 'drop_l5': 0.37280911323750693, 'drop_l6': 0.5139839908290262, 'drop_l7': 0.47988796751201396, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:32:23,215] Trial 789 finished with value: 0.4146941304206848 and parameters: {'drop_l1': 0.16838960086377922, 'drop_l2': 0.22588863551869182, 'drop_l3': 0.16147911641721766, 'drop_l4': 0.6657058981520396, 'drop_l5': 0.45744624113331345, 'drop_l6': 0.12111668267554888, 'drop_l7': 0.4700569262445643, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:32:23,227] Trial 787 finished with value: 0.43985414505004883 and parameters: {'drop_l1': 0.133411011656966, 'drop_l2': 0.23249242732241865, 'drop_l3': 0.21349519275019296, 'drop_l4': 0.676242935176048, 'drop_l5': 0.36753082242864266, 'drop_l6': 0.11792645542510939, 'drop_l7': 0.4622811193125072, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:32:55,784] Trial 790 finished with value: 0.42973512411117554 and parameters: {'drop_l1': 0.15888084723287937, 'drop_l2': 0.22881995469643882, 'drop_l3': 0.21298754420543153, 'drop_l4': 0.7334792364171534, 'drop_l5': 0.45825016920228545, 'drop_l6': 0.1163073419113797, 'drop_l7': 0.47245054037878037, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:00,191] Trial 791 finished with value: 0.42902952432632446 and parameters: {'drop_l1': 0.16597330998587112, 'drop_l2': 0.2331692779554619, 'drop_l3': 0.23962134560782414, 'drop_l4': 0.6521025453073341, 'drop_l5': 0.45658346103447084, 'drop_l6': 0.31827812897626095, 'drop_l7': 0.4600485326597602, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:01,930] Trial 793 finished with value: 0.4131088852882385 and parameters: {'drop_l1': 0.16519433121197566, 'drop_l2': 0.3380663326142135, 'drop_l3': 0.6505890561490844, 'drop_l4': 0.7388478454343435, 'drop_l5': 0.4180127414069317, 'drop_l6': 0.13508515963865214, 'drop_l7': 0.3430031948874077, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:01,969] Trial 792 finished with value: 0.43209028244018555 and parameters: {'drop_l1': 0.4799027824179263, 'drop_l2': 0.3332848440082094, 'drop_l3': 0.23822644976401777, 'drop_l4': 0.6492471043925285, 'drop_l5': 0.41929445901659845, 'drop_l6': 0.1340656264980336, 'drop_l7': 0.3429745066538915, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:31,346] Trial 794 finished with value: 0.42204204201698303 and parameters: {'drop_l1': 0.17572958985150566, 'drop_l2': 0.3290416773185833, 'drop_l3': 0.15332494495669619, 'drop_l4': 0.6513012551367021, 'drop_l5': 0.421192844679309, 'drop_l6': 0.13654668601673847, 'drop_l7': 0.38864544838108694, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:38,309] Trial 795 finished with value: 0.4398994445800781 and parameters: {'drop_l1': 0.4296807591806634, 'drop_l2': 0.3283689740515604, 'drop_l3': 0.309717555051641, 'drop_l4': 0.7123660812603766, 'drop_l5': 0.3881853784513777, 'drop_l6': 0.10087021791771444, 'drop_l7': 0.3449111570075771, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:39,567] Trial 797 finished with value: 0.434762179851532 and parameters: {'drop_l1': 0.11938002093646677, 'drop_l2': 0.35943209283717026, 'drop_l3': 0.7090747501814294, 'drop_l4': 0.5631955928092586, 'drop_l5': 0.38738688403697935, 'drop_l6': 0.17093616043002424, 'drop_l7': 0.38825705407810723, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:40,058] Trial 796 finished with value: 0.4621867537498474 and parameters: {'drop_l1': 0.12044361877408997, 'drop_l2': 0.29989743770834304, 'drop_l3': 0.3060710715071752, 'drop_l4': 0.5021016981865675, 'drop_l5': 0.3881015253108416, 'drop_l6': 0.17216703598560057, 'drop_l7': 0.38207920403279383, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:33:58,948] Trial 798 finished with value: 0.4323959946632385 and parameters: {'drop_l1': 0.1231687878414061, 'drop_l2': 0.35770330996078836, 'drop_l3': 0.2926067454432191, 'drop_l4': 0.7109164257587595, 'drop_l5': 0.3862882100167913, 'drop_l6': 0.17089270720510197, 'drop_l7': 0.43763817442286745, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874. [I 2021-03-10 19:34:03,588] Trial 799 finished with value: 0.40925461053848267 and parameters: {'drop_l1': 0.11783287493864107, 'drop_l2': 0.2667247093851606, 'drop_l3': 0.1418676227763072, 'drop_l4': 0.493366727894315, 'drop_l5': 0.3444928600977254, 'drop_l6': 0.17004270560620616, 'drop_l7': 0.43086279252527027, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 513 with value: 0.3998320400714874.
study_coarse.best_value, study_coarse.best_params
(0.3998320400714874,
{'drop_l1': 0.15872829858660079,
'drop_l2': 0.124404021283663,
'drop_l3': 0.1191444759931971,
'drop_l4': 0.6775070097550181,
'drop_l5': 0.32337913815894676,
'drop_l6': 0.13903817315530098,
'drop_l7': 0.39426663755264946,
'activation': 'relu',
'kernel_init': 'uniform'})
hp_score_board = hp_score_board.append({'Phase':'Coarse Tuning',
'Best Value':study_coarse.best_value,
'Hyperparameters':f'{study_coarse.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.398870 | {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256} |
| 1 | Coarse Tuning | 0.399832 | {'drop_l1': 0.15872829858660079, 'drop_l2': 0.124404021283663, 'drop_l3': 0.1191444759931971, 'drop_l4': 0.6775070097550181, 'drop_l5': 0.32337913815894676, 'drop_l6': 0.13903817315530098, 'drop_l7': 0.39426663755264946, 'activation': 'relu', 'kernel_init': 'uniform'} |
0.001 with relu as the activation function and uniform weight initialiser.study_coarse.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_activation | params_drop_l1 | params_drop_l2 | params_drop_l3 | params_drop_l4 | params_drop_l5 | params_drop_l6 | params_drop_l7 | params_kernel_init | state | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.642469 | 2021-03-10 17:16:34.645158 | 2021-03-10 17:17:07.186461 | 0 days 00:00:32.541303 | tanh | 0.192384 | 0.154283 | 0.343333 | 0.652803 | 0.850231 | 0.216713 | 0.368237 | he_normal | COMPLETE |
| 1 | 1 | 0.836918 | 2021-03-10 17:16:34.646346 | 2021-03-10 17:17:07.207396 | 0 days 00:00:32.561050 | tanh | 0.298693 | 0.429173 | 0.798145 | 0.606106 | 0.443316 | 0.263863 | 0.551602 | normal | COMPLETE |
| 2 | 2 | 0.627800 | 2021-03-10 17:16:34.650699 | 2021-03-10 17:17:07.218605 | 0 days 00:00:32.567906 | relu | 0.140323 | 0.722778 | 0.861971 | 0.634254 | 0.493032 | 0.161503 | 0.541625 | lecun_normal | COMPLETE |
| 3 | 3 | 0.572935 | 2021-03-10 17:16:34.651505 | 2021-03-10 17:17:07.179135 | 0 days 00:00:32.527630 | tanh | 0.844068 | 0.485660 | 0.147547 | 0.304336 | 0.474530 | 0.233403 | 0.303077 | normal | COMPLETE |
| 4 | 4 | 0.665749 | 2021-03-10 17:17:07.189239 | 2021-03-10 17:17:39.390242 | 0 days 00:00:32.201003 | relu | 0.450854 | 0.886925 | 0.871556 | 0.689466 | 0.121682 | 0.483609 | 0.398094 | normal | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 795 | 795 | 0.439899 | 2021-03-10 19:33:00.207098 | 2021-03-10 19:33:38.308746 | 0 days 00:00:38.101648 | relu | 0.429681 | 0.328369 | 0.309718 | 0.712366 | 0.388185 | 0.100870 | 0.344911 | uniform | COMPLETE |
| 796 | 796 | 0.462187 | 2021-03-10 19:33:01.963922 | 2021-03-10 19:33:40.058214 | 0 days 00:00:38.094292 | relu | 0.120444 | 0.299897 | 0.306071 | 0.502102 | 0.388102 | 0.172167 | 0.382079 | uniform | COMPLETE |
| 797 | 797 | 0.434762 | 2021-03-10 19:33:01.972827 | 2021-03-10 19:33:39.566856 | 0 days 00:00:37.594029 | relu | 0.119380 | 0.359432 | 0.709075 | 0.563196 | 0.387387 | 0.170936 | 0.388257 | uniform | COMPLETE |
| 798 | 798 | 0.432396 | 2021-03-10 19:33:31.354589 | 2021-03-10 19:33:58.947775 | 0 days 00:00:27.593186 | relu | 0.123169 | 0.357703 | 0.292607 | 0.710916 | 0.386288 | 0.170893 | 0.437638 | uniform | COMPLETE |
| 799 | 799 | 0.409255 | 2021-03-10 19:33:38.314203 | 2021-03-10 19:34:03.588051 | 0 days 00:00:25.273848 | relu | 0.117833 | 0.266725 | 0.141868 | 0.493367 | 0.344493 | 0.170043 | 0.430863 | uniform | COMPLETE |
800 rows × 15 columns
optuna.visualization.plot_parallel_coordinate(study_coarse)
optuna.visualization.plot_optimization_history(study_coarse)
optuna.visualization.plot_slice(study_coarse)
coarse_best_params = study_coarse.best_params
coarse_best_params, study_coarse.best_value
({'drop_l1': 0.15872829858660079,
'drop_l2': 0.124404021283663,
'drop_l3': 0.1191444759931971,
'drop_l4': 0.6775070097550181,
'drop_l5': 0.32337913815894676,
'drop_l6': 0.13903817315530098,
'drop_l7': 0.39426663755264946,
'activation': 'relu',
'kernel_init': 'uniform'},
0.3998320400714874)
drop_rate_per_layer =[coarse_best_params[f'drop_l{i}'] for i in range(1,arch_best_params['num_layers']+1)]
drop_rate_per_layer
coarse_best_params['drop_rate_per_layer'] = drop_rate_per_layer
for i in range(1,arch_best_params['num_layers']+1):
if f'drop_l{i}' in coarse_best_params.keys():
del coarse_best_params[f'drop_l{i}']
else:
pass
display(Markdown("#### Modified best parameters after Coarse Tuning :"))
print(coarse_best_params)
hp_best_params = {**arch_best_params , **coarse_best_params}
display(Markdown("#### Consolidate best parameters so far :"))
print(hp_best_params)
{'activation': 'relu', 'kernel_init': 'uniform', 'drop_rate_per_layer': [0.15872829858660079, 0.124404021283663, 0.1191444759931971, 0.6775070097550181, 0.32337913815894676, 0.13903817315530098, 0.39426663755264946]}
{'num_layers': 7, 'arch': 'D A B', 'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256], 'activation': 'relu', 'kernel_init': 'uniform', 'drop_rate_per_layer': [0.15872829858660079, 0.124404021283663, 0.1191444759931971, 0.6775070097550181, 0.32337913815894676, 0.13903817315530098, 0.39426663755264946]}
def create_optimizer(trial):
# We optimize the choice of optimizers as well as their parameters.
kwargs = {}
optimizer_options = ["RMSprop", "Adam", "SGD", "Adadelta"]
optimizer_selected = trial.suggest_categorical("optimizer", optimizer_options)
if optimizer_selected == "RMSprop":
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs["rho"] = trial.suggest_float("rho", 0.85, 0.99)
kwargs["momentum"] = trial.suggest_float("momentum", 1e-5, 1e-1, log=True)
kwargs['epsilon'] = trial.suggest_float("epsilon",1e-9, 1e-2, log=True)
elif optimizer_selected == "Adam":
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs['epsilon'] = trial.suggest_float("epsilon",1e-9, 1e-2, log=True)
kwargs['beta_1'] = trial.suggest_float("beta_1", 0.85, 0.99)
kwargs['beta_2'] = trial.suggest_float("beta_2", 0.85, 0.99)
elif optimizer_selected == "SGD":
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs["momentum"] = trial.suggest_float("momentum", 1e-5, 1e-1, log=True)
kwargs["nesterov"] = trial.suggest_categorical("nesterov",[True,False])
elif optimizer_selected == 'Adadelta':
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs["rho"] = trial.suggest_float("rho", 0.85,0.99)
optimizer = getattr(tf.optimizers, optimizer_selected)(**kwargs)
return optimizer
def objective_fine(trial):
global hp_best_params,loss,metric,X_train,X_test,y_train,y_test
epochs = 50
optimizer = create_optimizer(trial)
kwargs = {'opts': optimizer,
'n_epochs' : epochs
}
kwargs = {**hp_best_params, **kwargs}
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1,**kwargs)
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[0]
print(X_train.shape,X_test.shape,y_train.shape,y_test.shape)
study_fine = optuna.create_study(direction='minimize',sampler=optuna.samplers.TPESampler())
study_fine.optimize(objective_fine,n_trials=600,n_jobs=-13)
[I 2021-03-10 19:34:04,669] A new study created in memory with name: no-name-4eff942e-c742-4152-bb27-b122ab7b7f9c [I 2021-03-10 19:34:44,048] Trial 3 finished with value: 0.4963510036468506 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00015001162934165068, 'epsilon': 3.7737011096795016e-07, 'beta_1': 0.9116626516251398, 'beta_2': 0.864105652965245}. Best is trial 3 with value: 0.4963510036468506. [I 2021-03-10 19:34:46,235] Trial 0 finished with value: 0.6694058179855347 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.08772554190469146, 'epsilon': 6.102473768105114e-07, 'beta_1': 0.9856448754521859, 'beta_2': 0.9757298724746163}. Best is trial 3 with value: 0.4963510036468506. [I 2021-03-10 19:34:46,374] Trial 2 finished with value: 0.6371113061904907 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.018972407341984562, 'rho': 0.9104402709105183}. Best is trial 3 with value: 0.4963510036468506. [I 2021-03-10 19:34:46,408] Trial 1 finished with value: 0.4310475289821625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005178005893599341, 'epsilon': 0.0001496667012412544, 'beta_1': 0.8518927684486682, 'beta_2': 0.9673455827000104}. Best is trial 1 with value: 0.4310475289821625. [I 2021-03-10 19:35:24,065] Trial 4 finished with value: 0.4469136595726013 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005999792396832829, 'epsilon': 1.9034944490287042e-07, 'beta_1': 0.9218032424503257, 'beta_2': 0.896629951978047}. Best is trial 1 with value: 0.4310475289821625. [I 2021-03-10 19:35:24,561] Trial 5 finished with value: 5.222539901733398 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00012236064376983991, 'momentum': 1.2972963715697444e-05, 'nesterov': False}. Best is trial 1 with value: 0.4310475289821625. [I 2021-03-10 19:35:26,837] Trial 6 finished with value: 4.883284091949463 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.004695160332776656, 'rho': 0.9557080845342952}. Best is trial 1 with value: 0.4310475289821625. [I 2021-03-10 19:35:26,997] Trial 7 finished with value: 0.42949312925338745 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.05354679444915394, 'momentum': 0.0018104142972969136, 'nesterov': True}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:05,101] Trial 8 finished with value: 5.424927711486816 and parameters: {'optimizer': 'SGD', 'learning_rate': 2.4846773306998327e-05, 'momentum': 0.003075379526090657, 'nesterov': False}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:05,754] Trial 9 finished with value: 5.69367790222168 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0002082943829079414, 'rho': 0.9264952591750488}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:07,114] Trial 10 finished with value: 5.500489711761475 and parameters: {'optimizer': 'SGD', 'learning_rate': 3.401479515177971e-05, 'momentum': 8.310006593818712e-05, 'nesterov': True}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:07,611] Trial 11 finished with value: 0.4331551492214203 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0019812012358484377, 'epsilon': 4.106510902154763e-05, 'beta_1': 0.9647482677207568, 'beta_2': 0.9539295500612015}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:48,394] Trial 12 finished with value: 5.528838157653809 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0011189774508283518, 'rho': 0.9754236375784352}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:49,081] Trial 13 finished with value: 21171.99609375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.07736101077132053, 'rho': 0.8549723659646804, 'momentum': 0.0962201530165217, 'epsilon': 1.1867145644106935e-09}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:51,619] Trial 14 finished with value: 0.522510290145874 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.003794577119154257, 'rho': 0.8508063747359814, 'momentum': 0.018747430544982905, 'epsilon': 0.0024475848635793123}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:36:51,622] Trial 15 finished with value: 1.0173214673995972 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.09305627088560077, 'rho': 0.8512635196671388, 'momentum': 0.09382930027982775, 'epsilon': 0.0014955950086159105}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:37:31,383] Trial 17 finished with value: 0.4331343173980713 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.009875935907385822, 'momentum': 0.006766157199390071, 'nesterov': True}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:37:32,419] Trial 16 finished with value: 1.1856426000595093 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.06896284944509873, 'rho': 0.860006394068258, 'momentum': 0.059318924614289205, 'epsilon': 0.009140896339371537}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:37:33,608] Trial 19 finished with value: 0.43587785959243774 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.01295103644314912, 'momentum': 0.0006456277273952202, 'nesterov': True}. Best is trial 7 with value: 0.42949312925338745. [I 2021-03-10 19:37:33,955] Trial 18 finished with value: 0.41924887895584106 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.014887038534359846, 'momentum': 0.00047937039855896053, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:14,625] Trial 20 finished with value: 3.9879508018493652 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00042134857719756995, 'momentum': 0.0004145068431142445, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:16,825] Trial 23 finished with value: 0.5233125686645508 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.029102059123537534, 'momentum': 0.0003721004135471857, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:17,027] Trial 22 finished with value: 0.47406595945358276 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004449350765825809, 'epsilon': 8.657972900056956e-05, 'beta_1': 0.8560531990880261, 'beta_2': 0.9899153843819741}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:17,046] Trial 21 finished with value: 0.4204021394252777 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00047867833292832744, 'epsilon': 5.454175977713677e-05, 'beta_1': 0.8536042626653684, 'beta_2': 0.9366397889809613}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:53,954] Trial 24 finished with value: 0.43395358324050903 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.027519743611437955, 'momentum': 0.00014870404165490489, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:57,116] Trial 26 finished with value: 0.45167574286460876 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005604440613583421, 'momentum': 5.582672112738968e-05, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:57,309] Trial 25 finished with value: 6.957940101623535 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.031675230299682555, 'epsilon': 3.631934769216964e-05, 'beta_1': 0.8614822222216287, 'beta_2': 0.9882838045655253}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:38:57,358] Trial 27 finished with value: 0.5255135297775269 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.03370612776291878, 'momentum': 6.43265411804309e-05, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:39:30,348] Trial 28 finished with value: 0.4213863015174866 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006537884738746893, 'momentum': 0.0018445494051290675, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:39:36,785] Trial 29 finished with value: 5.208498477935791 and parameters: {'optimizer': 'SGD', 'learning_rate': 5.5067586335979796e-05, 'momentum': 0.0021824070890254493, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:39:37,119] Trial 30 finished with value: 5.398308753967285 and parameters: {'optimizer': 'SGD', 'learning_rate': 6.144771596106121e-05, 'momentum': 0.0026803083104242603, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:39:37,177] Trial 31 finished with value: 5.248342037200928 and parameters: {'optimizer': 'SGD', 'learning_rate': 5.964153109009725e-05, 'momentum': 0.0019946918959451213, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:39:59,633] Trial 32 finished with value: 0.4484017789363861 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0020530462269737907, 'epsilon': 5.3377712553358165e-09, 'beta_1': 0.8937298811262294, 'beta_2': 0.9246511828815018}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:17,465] Trial 34 finished with value: 0.4803120493888855 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0019198003919519292, 'momentum': 0.0013588796924748488, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:17,535] Trial 35 finished with value: 0.46659842133522034 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0021185559189378957, 'epsilon': 3.5574004370642802e-09, 'beta_1': 0.8939648487171526, 'beta_2': 0.9283364001222926}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:17,578] Trial 33 finished with value: 5.41737699508667 and parameters: {'optimizer': 'Adam', 'learning_rate': 1.0643465616256281e-05, 'epsilon': 8.417499499501427e-09, 'beta_1': 0.8971722687590515, 'beta_2': 0.9274860249427311}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:28,831] Trial 36 finished with value: 0.4236694872379303 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0097939529419927, 'momentum': 0.01208527898248232, 'nesterov': True}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:57,613] Trial 38 finished with value: 0.42845138907432556 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008947497041662672, 'momentum': 0.010501984826429837, 'nesterov': False}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:58,464] Trial 37 finished with value: 229.3902587890625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.007577492325209697, 'epsilon': 3.5408497102039894e-06, 'beta_1': 0.8847280072378014, 'beta_2': 0.9305880162863148}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:40:58,486] Trial 39 finished with value: 0.42201489210128784 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007616135359806015, 'momentum': 0.013989463407351911, 'nesterov': False}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:41:03,379] Trial 40 finished with value: 0.42326149344444275 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007732334293288217, 'momentum': 0.010997686658731545, 'nesterov': False}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:41:39,352] Trial 41 finished with value: 0.6100424528121948 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.01820699670167058, 'rho': 0.8917371794323212}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:41:41,480] Trial 42 finished with value: 5.411915302276611 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0009132478352141741, 'rho': 0.9031637942233284}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:41:42,150] Trial 43 finished with value: 5.552046775817871 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0008936248442828539, 'rho': 0.8981263632371292}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:41:42,674] Trial 44 finished with value: 0.4692464768886566 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.016176402081191293, 'momentum': 0.034040125760446745, 'nesterov': False}. Best is trial 18 with value: 0.41924887895584106. [I 2021-03-10 19:42:19,104] Trial 45 finished with value: 0.4141204357147217 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0034788421972414716, 'momentum': 0.03330451322285664, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:42:23,055] Trial 46 finished with value: 0.43737879395484924 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0032575496421982466, 'momentum': 0.033058446426760216, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:42:23,409] Trial 47 finished with value: 0.4228224754333496 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.015579481374759457, 'momentum': 0.030873076269995536, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:42:23,426] Trial 48 finished with value: 0.4466533660888672 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003387518601959751, 'momentum': 0.005002064316903014, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:42:54,995] Trial 49 finished with value: 0.4245052933692932 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0030230438710372324, 'momentum': 0.029503425976272087, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:02,876] Trial 50 finished with value: 0.44337454438209534 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005044847736468779, 'momentum': 0.0052747026706626565, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:04,750] Trial 51 finished with value: 4.713891506195068 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0003102826070822663, 'momentum': 0.0007874245018291953, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:04,789] Trial 52 finished with value: 0.4855995774269104 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00020030980771715024, 'epsilon': 4.81291889876307e-06, 'beta_1': 0.9387096337491371, 'beta_2': 0.8871805404679286}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:25,181] Trial 53 finished with value: 0.4148707389831543 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012573164590408707, 'epsilon': 5.258949748329869e-06, 'beta_1': 0.9403206962331865, 'beta_2': 0.8950916203156866}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:43,044] Trial 54 finished with value: 4.741097450256348 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0002270881395161705, 'momentum': 0.04994877701007342, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:45,864] Trial 55 finished with value: 0.5191994309425354 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0013320900319364975, 'momentum': 0.06355872123260213, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:45,949] Trial 56 finished with value: 0.5644882321357727 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.05406606784227751, 'momentum': 0.05624625050839868, 'nesterov': False}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:43:56,767] Trial 57 finished with value: 0.4239298701286316 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006328248544134878, 'epsilon': 0.0007570982268502065, 'beta_1': 0.9492973982598224, 'beta_2': 0.9021256465678025}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:44:22,802] Trial 58 finished with value: 0.4219796061515808 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0015463241402404531, 'epsilon': 5.064642965888537e-08, 'beta_1': 0.9587617197670036, 'beta_2': 0.9001656032301227}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:44:28,593] Trial 59 finished with value: 0.4169433116912842 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007103945734338112, 'epsilon': 0.0003184937291331792, 'beta_1': 0.9529206912752981, 'beta_2': 0.8806696467332346}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:44:28,760] Trial 60 finished with value: 0.43797439336776733 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007104607831927455, 'epsilon': 0.00043301843582215047, 'beta_1': 0.9477231264956719, 'beta_2': 0.897542996969436}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:44:33,947] Trial 61 finished with value: 0.5607534646987915 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001250495163895254, 'epsilon': 3.570773648447153e-08, 'beta_1': 0.9754109023884154, 'beta_2': 0.8584189161343012}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:03,194] Trial 62 finished with value: 0.4555361866950989 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0015362399901291059, 'epsilon': 8.238847308219825e-08, 'beta_1': 0.9695594175750034, 'beta_2': 0.8730355935448201}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:09,831] Trial 63 finished with value: 0.4294326901435852 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005772272606109625, 'epsilon': 1.1582742644502362e-05, 'beta_1': 0.9810698408262785, 'beta_2': 0.8718619371246107}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:10,836] Trial 64 finished with value: 0.49422115087509155 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00131737466363337, 'epsilon': 1.3315736779003827e-05, 'beta_1': 0.9689856166771327, 'beta_2': 0.8732761080510536}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:12,678] Trial 65 finished with value: 0.441969633102417 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0015290436523687301, 'epsilon': 1.2465085087757078e-05, 'beta_1': 0.9617069611849278, 'beta_2': 0.880927364408923}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:34,738] Trial 66 finished with value: 0.4223461151123047 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003320635213709773, 'epsilon': 1.1379778241171668e-05, 'beta_1': 0.9318597882965721, 'beta_2': 0.9088307925044269}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:50,878] Trial 67 finished with value: 0.41744905710220337 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0024571285722685006, 'epsilon': 1.3457763204610875e-05, 'beta_1': 0.9346281809145066, 'beta_2': 0.9127882963338201}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:52,168] Trial 68 finished with value: 0.4376121163368225 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042268918176639503, 'epsilon': 0.00021056071373896104, 'beta_1': 0.9399609189945926, 'beta_2': 0.9092326588121817}. Best is trial 45 with value: 0.4141204357147217. [I 2021-03-10 19:45:52,827] Trial 69 finished with value: 0.4006522595882416 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003505227759262581, 'rho': 0.9853377164336544, 'momentum': 0.00018123906046506566, 'epsilon': 1.347199795487488e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:46:06,380] Trial 70 finished with value: 1.2324029207229614 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002525860279034927, 'rho': 0.9839080954767735, 'momentum': 1.5094981830497268e-05, 'epsilon': 1.3642701728082173e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:46:30,026] Trial 73 finished with value: 0.7112628221511841 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002489906316504126, 'rho': 0.9888728795871942, 'momentum': 2.051650045149682e-05, 'epsilon': 1.0915624162439615e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:46:30,031] Trial 72 finished with value: 0.6675187349319458 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0061481862925632565, 'rho': 0.9892514741383641, 'momentum': 1.3481904745931318e-05, 'epsilon': 3.598485930260907e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:46:30,068] Trial 71 finished with value: 0.5458618402481079 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002377996819765432, 'rho': 0.9845707525430032, 'momentum': 0.00013479434604949303, 'epsilon': 1.4993988348498428e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:46:36,382] Trial 74 finished with value: 0.545110821723938 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001240608058652668, 'rho': 0.946689242863353, 'momentum': 0.0001662016648265126, 'epsilon': 1.2664660705598988e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:12,411] Trial 76 finished with value: 0.5212848782539368 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00012516181201971199, 'rho': 0.9501247469738385, 'momentum': 0.00021134715961821634, 'epsilon': 0.00012028558744647446}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:12,534] Trial 75 finished with value: 0.49540823698043823 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.004086745048969695, 'rho': 0.9463175908964184, 'momentum': 0.00017109402547140688, 'epsilon': 0.00011138379832993242}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:12,564] Trial 77 finished with value: 0.5650612711906433 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.012250726391249732, 'rho': 0.9464230107078859, 'momentum': 0.00025947697423977313, 'epsilon': 4.20819419386173e-07}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:14,412] Trial 78 finished with value: 0.42228609323501587 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008210222950150267, 'epsilon': 0.000108991752706419, 'beta_1': 0.9245633377800936, 'beta_2': 0.9372767463155398}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:52,722] Trial 79 finished with value: 0.43201351165771484 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007445946937872349, 'epsilon': 3.95825846241626e-07, 'beta_1': 0.9202569059984012, 'beta_2': 0.9450671482907358}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:54,992] Trial 81 finished with value: 0.4304329752922058 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007744905020675909, 'epsilon': 4.689420117288116e-06, 'beta_1': 0.9240572375911884, 'beta_2': 0.9461101624157169}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:56,009] Trial 80 finished with value: 0.4329516291618347 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008392950518405998, 'epsilon': 5.320290032447201e-06, 'beta_1': 0.9226899236794383, 'beta_2': 0.9447804273684394}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:47:56,019] Trial 82 finished with value: 0.44305744767189026 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010157343682738128, 'epsilon': 6.806934717399648e-06, 'beta_1': 0.9124481117254978, 'beta_2': 0.9429112759382242}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:48:33,716] Trial 83 finished with value: 0.6642266511917114 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.023057420085474793, 'epsilon': 5.055047361325847e-06, 'beta_1': 0.9095296126169293, 'beta_2': 0.9178757185509809}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:48:38,529] Trial 86 finished with value: 0.44232624769210815 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005289457585474502, 'epsilon': 3.479462714477153e-05, 'beta_1': 0.9512743596700306, 'beta_2': 0.9141541240465367}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:48:39,092] Trial 84 finished with value: 3696.001220703125 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.022626874360506314, 'epsilon': 0.00033558243449941254, 'beta_1': 0.9520303713048708, 'beta_2': 0.9118358351941309}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:48:39,190] Trial 85 finished with value: 0.4502675533294678 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000538571617077875, 'epsilon': 3.183980489514467e-05, 'beta_1': 0.9557107981566129, 'beta_2': 0.8878115861265746}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:49:12,197] Trial 87 finished with value: 0.43340855836868286 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003205593383477397, 'epsilon': 3.348656749268753e-05, 'beta_1': 0.9574452695062873, 'beta_2': 0.8894881712578685}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:49:22,536] Trial 88 finished with value: 0.4153873324394226 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0017130350017903085, 'epsilon': 4.674328026323554e-08, 'beta_1': 0.9567203369356614, 'beta_2': 0.8883510526872044}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:49:22,599] Trial 90 finished with value: 5.433112144470215 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0003290209683751414, 'rho': 0.8732450670564611}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:49:22,618] Trial 89 finished with value: 0.4506978392601013 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003129308337841302, 'epsilon': 2.0634964543443947e-07, 'beta_1': 0.9887116531744292, 'beta_2': 0.8888850958590483}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:49:45,356] Trial 91 finished with value: 5.39560079574585 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0018232202579334523, 'rho': 0.8742717819534256}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:06,639] Trial 94 finished with value: 0.5054429769515991 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001979312982133775, 'epsilon': 2.7206865916503096e-08, 'beta_1': 0.9405884231193377, 'beta_2': 0.9027448913904756}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:06,726] Trial 93 finished with value: 0.5240576863288879 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.004139011603739328, 'epsilon': 1.2177295390676427e-08, 'beta_1': 0.9393093729332754, 'beta_2': 0.901924694745462}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:06,852] Trial 92 finished with value: 5.239664554595947 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0042553289881170474, 'rho': 0.9688439217954531}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:20,694] Trial 95 finished with value: 0.48495131731033325 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.004296015744804027, 'epsilon': 3.5064211330489075e-08, 'beta_1': 0.939586795280783, 'beta_2': 0.8979390784609982}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:52,308] Trial 96 finished with value: 1.3910354375839233 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00396982400771333, 'epsilon': 1.3524973998687842e-08, 'beta_1': 0.942589648870061, 'beta_2': 0.8970822568326724}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:53,044] Trial 98 finished with value: 0.4384986460208893 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0028720223228729906, 'epsilon': 0.003198387246834072, 'beta_1': 0.9627822790151853, 'beta_2': 0.8797934906472706}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:53,117] Trial 97 finished with value: 0.45057621598243713 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010364914244312623, 'epsilon': 1.0438232118413546e-07, 'beta_1': 0.9320216409202998, 'beta_2': 0.9637783179843267}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:50:58,497] Trial 99 finished with value: 0.4489821791648865 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0031063511722117746, 'epsilon': 0.0032477280804231495, 'beta_1': 0.9310570635420936, 'beta_2': 0.880940279508616}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:51:35,695] Trial 100 finished with value: 0.4257291853427887 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005922337134203734, 'momentum': 3.006885721086605e-05, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:51:38,543] Trial 101 finished with value: 0.5075143575668335 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0015774651296637718, 'momentum': 3.600570333232967e-05, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:51:38,556] Trial 102 finished with value: 0.43741387128829956 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005563982396577082, 'momentum': 4.1514593699850617e-05, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:51:40,519] Trial 103 finished with value: 0.40101689100265503 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006136090293513125, 'momentum': 0.00042201337375815046, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:52:17,900] Trial 104 finished with value: 0.4415643811225891 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0073242250362569716, 'momentum': 0.00045046183025919514, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:52:23,911] Trial 106 finished with value: 0.4460532069206238 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007584709696896764, 'momentum': 0.00048209914618574384, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:52:23,965] Trial 105 finished with value: 0.42588311433792114 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007662099007124735, 'momentum': 0.000591061067358385, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:52:24,000] Trial 107 finished with value: 0.4551076889038086 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.012873384993560283, 'momentum': 0.0004590220576931018, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:52:54,463] Trial 108 finished with value: 0.5369588732719421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0011872214322570576, 'momentum': 0.0011400375611080754, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:05,294] Trial 111 finished with value: 5.099721431732178 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00023277498525443803, 'momentum': 0.00010118737595746013, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:05,822] Trial 110 finished with value: 0.4570365846157074 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.013162937103643163, 'momentum': 0.0001033777391120726, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:06,002] Trial 109 finished with value: 4.89997673034668 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00024753082975706333, 'momentum': 0.0011030236809346998, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:24,421] Trial 112 finished with value: 0.40957051515579224 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004158307573584442, 'epsilon': 2.4255260934705505e-06, 'beta_1': 0.8682648240575492, 'beta_2': 0.8534940889188243}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:45,512] Trial 114 finished with value: 0.4308278560638428 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.009658598802711976, 'momentum': 0.0002997052912851846, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:47,287] Trial 115 finished with value: 0.4254991412162781 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0024794890075116615, 'momentum': 0.00031956458393190873, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:47,301] Trial 113 finished with value: 0.42854008078575134 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004169909001722368, 'epsilon': 2.1696903350625246e-09, 'beta_1': 0.8771443229109456, 'beta_2': 0.9233731961028839}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:53:57,547] Trial 116 finished with value: 0.42489248514175415 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004481179147735544, 'epsilon': 2.0144041170636636e-06, 'beta_1': 0.8671161983997865, 'beta_2': 0.8501961223137683}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:54:28,627] Trial 117 finished with value: 0.4292599558830261 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00038207230063821653, 'epsilon': 6.171710798281956e-05, 'beta_1': 0.8646226861897586, 'beta_2': 0.8523349728093712}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:54:28,640] Trial 118 finished with value: 0.4504423141479492 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004163086680829125, 'epsilon': 6.135946467662897e-05, 'beta_1': 0.8516739364014283, 'beta_2': 0.8510004521138361}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:54:28,643] Trial 119 finished with value: 0.49225980043411255 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001892598165889311, 'epsilon': 2.344569244865696e-06, 'beta_1': 0.8526073528971276, 'beta_2': 0.8681199835715134}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:54:33,504] Trial 120 finished with value: 0.4164099097251892 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006515954233443803, 'epsilon': 5.949137617032394e-05, 'beta_1': 0.8576712190009077, 'beta_2': 0.8540007141605948}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:09,771] Trial 121 finished with value: 0.4366529583930969 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006517455759584182, 'epsilon': 0.0010545592435370302, 'beta_1': 0.9465614052535767, 'beta_2': 0.9051292335021861}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:11,517] Trial 122 finished with value: 0.46496349573135376 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016975429013950283, 'epsilon': 7.313045453979078e-07, 'beta_1': 0.9563403005527229, 'beta_2': 0.8605533599582402}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:12,007] Trial 124 finished with value: 0.4499344825744629 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006876541999645676, 'epsilon': 2.010604076219435e-05, 'beta_1': 0.8586059034380772, 'beta_2': 0.8634552978686734}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:12,235] Trial 123 finished with value: 0.46265655755996704 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0017271677826312397, 'epsilon': 6.049839812981214e-07, 'beta_1': 0.9455693852003899, 'beta_2': 0.8926004363612476}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:53,356] Trial 125 finished with value: 0.42928561568260193 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016418514917363714, 'epsilon': 2.0266355344682443e-05, 'beta_1': 0.8593672166975099, 'beta_2': 0.8624708341255495}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:55,014] Trial 127 finished with value: 0.565754234790802 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.03766087282660521, 'epsilon': 2.0497351516528627e-05, 'beta_1': 0.87069551623342, 'beta_2': 0.9155122672057417}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:55,076] Trial 126 finished with value: 0.4366535544395447 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006368767641825526, 'epsilon': 1.6498574530942715e-05, 'beta_1': 0.8588723214505581, 'beta_2': 0.8931178552910887}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:55:55,092] Trial 128 finished with value: 0.4381694197654724 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0011517950181848683, 'rho': 0.9361768920099527, 'momentum': 0.0016275149870360768, 'epsilon': 0.0002551722446015722}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:56:34,977] Trial 129 finished with value: 0.45842891931533813 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.001157727685260747, 'rho': 0.9285348885062717, 'momentum': 0.0015228475142365966, 'epsilon': 0.00023902970776933165}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:56:37,935] Trial 132 finished with value: 0.45449766516685486 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.002187809654198799, 'epsilon': 7.969778844182892e-06, 'beta_1': 0.971316794750244, 'beta_2': 0.9315592615193308}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:56:38,027] Trial 131 finished with value: 0.516573429107666 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0021621361861746957, 'rho': 0.9625702699140897, 'momentum': 0.0032026140111114836, 'epsilon': 1.6749811712617232e-07}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:56:38,191] Trial 130 finished with value: 0.4346815049648285 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009562415481488163, 'rho': 0.9231208580351562, 'momentum': 0.0008374110056125912, 'epsilon': 0.00029890983499833776}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:57:15,835] Trial 133 finished with value: 0.5349715948104858 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001618595835964478, 'epsilon': 8.4592075894363e-06, 'beta_1': 0.9702750585748834, 'beta_2': 0.8828270760618036}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:57:18,087] Trial 136 finished with value: 0.41507816314697266 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.010716201299516354, 'momentum': 0.014235520036030333, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:57:20,030] Trial 134 finished with value: 3.6932525634765625 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005211818087849574, 'momentum': 0.017547348374160575, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:57:20,257] Trial 135 finished with value: 3.5495002269744873 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0004852173054830096, 'momentum': 0.09123294999854653, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:57:59,681] Trial 137 finished with value: 0.43134593963623047 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005194313094626478, 'momentum': 0.020414428303378012, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:01,841] Trial 138 finished with value: 0.43676090240478516 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.010847781019191738, 'momentum': 0.021482075562183903, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:01,892] Trial 139 finished with value: 0.46388086676597595 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0013604153147601797, 'momentum': 0.007920505105460768, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:02,994] Trial 140 finished with value: 0.5104689598083496 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.004920987756518166, 'epsilon': 0.0004855304132040672, 'beta_1': 0.9796347353649291, 'beta_2': 0.9216130936252934}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:42,341] Trial 141 finished with value: 0.509131133556366 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.011955183111120675, 'epsilon': 0.0005290476721075272, 'beta_1': 0.9343625697805685, 'beta_2': 0.8567578377358612}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:45,199] Trial 143 finished with value: 229621792.0 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.015924218225694748, 'epsilon': 3.15148276927058e-06, 'beta_1': 0.980623485899907, 'beta_2': 0.8563717979998557}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:45,245] Trial 144 finished with value: 0.49191388487815857 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.016082950230734895, 'momentum': 0.003964131655464938, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:58:45,286] Trial 142 finished with value: 0.4708811640739441 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.015237182176575102, 'epsilon': 3.224568111086332e-06, 'beta_1': 0.873425534647241, 'beta_2': 0.9216520070082339}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:59:21,560] Trial 145 finished with value: 0.4364999830722809 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.015494472454913323, 'momentum': 0.0038460018709300355, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:59:24,932] Trial 146 finished with value: 0.4226103723049164 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006490783824808923, 'momentum': 0.014194396973144613, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:59:26,287] Trial 147 finished with value: 0.4186915457248688 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.009185065069844508, 'momentum': 0.012422284278635226, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:59:26,849] Trial 148 finished with value: 0.42433977127075195 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006317439453503769, 'momentum': 0.014929914715144377, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 19:59:59,085] Trial 149 finished with value: 0.42707353830337524 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003266821804590042, 'momentum': 0.01553237016833283, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:05,558] Trial 150 finished with value: 0.42964285612106323 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008511174322045078, 'momentum': 0.007212549423127935, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:07,105] Trial 151 finished with value: 0.4136137068271637 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003501032266506693, 'momentum': 0.008077144715162316, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:07,121] Trial 152 finished with value: 0.46002721786499023 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0034999880080590967, 'momentum': 0.0006404277390630293, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:31,837] Trial 153 finished with value: 0.4490202069282532 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000259052496636831, 'epsilon': 6.54438507399352e-05, 'beta_1': 0.9546317316619306, 'beta_2': 0.9088404813324976}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:46,483] Trial 154 finished with value: 0.433711439371109 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.010080637180630835, 'momentum': 0.009164751684360336, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:49,631] Trial 155 finished with value: 0.4204714894294739 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003355655122172635, 'momentum': 0.008683199689424525, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:00:49,643] Trial 156 finished with value: 0.4272539019584656 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.002788406103223628, 'momentum': 0.0095804177515834, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:01:04,203] Trial 157 finished with value: 0.432792603969574 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0026727468949483387, 'momentum': 0.009930648446033212, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:01:25,862] Trial 158 finished with value: 1.0360805988311768 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0008703253267873686, 'momentum': 0.025784444108974652, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:01:31,599] Trial 159 finished with value: 0.4681505262851715 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0025902310189325176, 'momentum': 0.005788806130498095, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:01:31,601] Trial 160 finished with value: 0.42633146047592163 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.020058005026294255, 'momentum': 0.005451228072629903, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:01:38,817] Trial 161 finished with value: 0.42676740884780884 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0036934280424930355, 'momentum': 0.005285252804081857, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:00,606] Trial 162 finished with value: 0.4313080906867981 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.004844049356581137, 'momentum': 0.04414419312002613, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:12,974] Trial 163 finished with value: 0.4132828712463379 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0037103765953016643, 'momentum': 0.04768474280587753, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:13,016] Trial 164 finished with value: 0.42267656326293945 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003487600840183852, 'momentum': 0.002396979005323875, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:17,838] Trial 165 finished with value: 0.46059051156044006 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.004524594994091152, 'epsilon': 7.25973278089694e-08, 'beta_1': 0.9633418376992797, 'beta_2': 0.8744059344174798}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:32,635] Trial 166 finished with value: 0.4604824185371399 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003640391709844945, 'epsilon': 0.00018421576979667055, 'beta_1': 0.960372701199681, 'beta_2': 0.8758500157786553}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:54,973] Trial 168 finished with value: 0.41218167543411255 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008701711245350982, 'momentum': 0.09937983456145766, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:54,991] Trial 167 finished with value: 0.41567540168762207 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008710587370514111, 'momentum': 0.039519609350564644, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:02:57,227] Trial 169 finished with value: 0.4228593707084656 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0089767347896059, 'momentum': 0.06750915346351018, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:03:05,294] Trial 170 finished with value: 0.43475252389907837 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006559934496126191, 'momentum': 0.06725885615893348, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:03:37,616] Trial 172 finished with value: 0.4153541624546051 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008903217151594051, 'momentum': 0.07820977759363919, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:03:37,620] Trial 173 finished with value: 0.41869592666625977 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0072174441780743495, 'momentum': 0.03950930909416174, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:03:37,635] Trial 171 finished with value: 0.42418140172958374 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0083892446888582, 'momentum': 0.07508408530043935, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:03:42,044] Trial 174 finished with value: 0.43101033568382263 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.010809275926165526, 'momentum': 0.0809103429138169, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:04:19,453] Trial 175 finished with value: 0.44728580117225647 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.011156849347374424, 'momentum': 0.04698520170858888, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:04:20,654] Trial 177 finished with value: 0.4393465518951416 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.011349780399674912, 'momentum': 0.041067542750432263, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:04:20,732] Trial 176 finished with value: 0.4281066060066223 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008694728938084203, 'momentum': 0.041234414070012965, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:04:22,802] Trial 178 finished with value: 0.4177793562412262 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.012593151115089256, 'momentum': 0.0372005965232535, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:02,409] Trial 179 finished with value: 0.4241011142730713 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007975765742085387, 'momentum': 0.04131771793395585, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:03,869] Trial 181 finished with value: 0.4233417510986328 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007179939822996762, 'momentum': 0.09880063524555256, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:04,027] Trial 182 finished with value: 0.4298962652683258 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006888363157516977, 'momentum': 0.09833812112677077, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:04,192] Trial 180 finished with value: 0.4449199140071869 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0074979279616870925, 'momentum': 0.03830293026489601, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:43,582] Trial 183 finished with value: 0.4333623945713043 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005315148747519924, 'momentum': 0.027284559992012, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:44,329] Trial 186 finished with value: 0.4307492673397064 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005237826595390203, 'momentum': 0.024638832854904178, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:45,429] Trial 185 finished with value: 0.42722493410110474 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.013152463100559167, 'momentum': 0.024518042740332927, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:05:45,508] Trial 184 finished with value: 0.4219503402709961 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.013247411062471615, 'momentum': 0.05334990075805924, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:06:22,945] Trial 187 finished with value: 0.43744921684265137 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.01225381639468399, 'momentum': 0.053313380282248364, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:06:25,557] Trial 189 finished with value: 0.43928131461143494 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.009799907761432127, 'momentum': 0.018508642485380585, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:06:26,209] Trial 188 finished with value: 0.5967628359794617 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.012362765419059512, 'rho': 0.9080964493979554}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:06:26,613] Trial 190 finished with value: 1.1985163688659668 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.009361970973187363, 'rho': 0.9138651807134154}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:05,312] Trial 191 finished with value: 0.6663504838943481 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.009345845008687534, 'rho': 0.9109440136092782}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:05,377] Trial 193 finished with value: 0.4585978388786316 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.019453862756046224, 'momentum': 0.03307696860674038, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:05,912] Trial 192 finished with value: 0.43264245986938477 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00937092213652735, 'momentum': 0.0355358979155844, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:05,944] Trial 194 finished with value: 3.8560760021209717 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0004997296754533615, 'momentum': 0.03436606354518997, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:44,981] Trial 195 finished with value: 4.01569128036499 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00046697914793318286, 'momentum': 0.03405063808834955, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:47,912] Trial 196 finished with value: 0.4841737747192383 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00027013735425127674, 'rho': 0.9762907869892001, 'momentum': 0.0002511260409923108, 'epsilon': 0.00014422251768615615}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:48,026] Trial 198 finished with value: 0.473031610250473 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006210228225583224, 'rho': 0.9748437261092269, 'momentum': 0.0630460735694342, 'epsilon': 8.017835701973229e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:07:48,062] Trial 197 finished with value: 0.4482139050960541 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005311739643805614, 'rho': 0.971657749223996, 'momentum': 0.011448994254565024, 'epsilon': 8.220054360245736e-07}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:08:24,561] Trial 199 finished with value: 0.44797199964523315 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000607619369525051, 'rho': 0.8853356005020079, 'momentum': 0.012966183099582185, 'epsilon': 9.825635490044295e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:08:27,851] Trial 202 finished with value: 0.420012891292572 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007619158121368961, 'epsilon': 1.95127190711004e-06, 'beta_1': 0.9283507263350292, 'beta_2': 0.9339893699157505}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:08:28,865] Trial 201 finished with value: 0.8562062978744507 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.023300584380031766, 'epsilon': 4.9841890850970696e-05, 'beta_1': 0.9156601053568731, 'beta_2': 0.9546570017354576}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:08:28,893] Trial 200 finished with value: 0.4471700191497803 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006867674153853073, 'epsilon': 9.187263695621558e-07, 'beta_1': 0.9013670025923711, 'beta_2': 0.9570305325345916}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:07,878] Trial 203 finished with value: 0.4344797134399414 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.005860935131765519, 'epsilon': 1.0551138685577743e-05, 'beta_1': 0.9350170659203498, 'beta_2': 0.9368903504868364}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:09,566] Trial 204 finished with value: 0.421518474817276 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007688972829488135, 'epsilon': 2.340601253649341e-06, 'beta_1': 0.9173239445818777, 'beta_2': 0.9412344597873781}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:11,481] Trial 206 finished with value: 0.43889322876930237 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008281918231012676, 'epsilon': 2.0390018295407925e-06, 'beta_1': 0.9348946391078031, 'beta_2': 0.937545568328447}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:11,520] Trial 205 finished with value: 0.43928012251853943 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009474156529276116, 'epsilon': 2.6428048806688787e-07, 'beta_1': 0.9266430546153456, 'beta_2': 0.9344309520281828}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:49,783] Trial 207 finished with value: 0.48360952734947205 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007609658076626007, 'epsilon': 1.5175922085611902e-06, 'beta_1': 0.9295006193045883, 'beta_2': 0.8680537046965788}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:51,775] Trial 208 finished with value: 0.42030414938926697 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008307236440821744, 'epsilon': 2.7697350199203325e-07, 'beta_1': 0.9265340301009529, 'beta_2': 0.9338039345663351}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:54,206] Trial 210 finished with value: 0.44222623109817505 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013948379848803271, 'epsilon': 5.4995667425086e-07, 'beta_1': 0.8822332024077201, 'beta_2': 0.8674056801973212}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:09:54,251] Trial 209 finished with value: 0.6223102807998657 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0020652914612660916, 'epsilon': 2.8465689188389027e-07, 'beta_1': 0.8867207533622367, 'beta_2': 0.9275626125622748}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:10:29,459] Trial 211 finished with value: 0.4297439455986023 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0042073781541207, 'momentum': 0.08053200970343595, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:10:33,232] Trial 212 finished with value: 0.5077149868011475 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0014477005166384134, 'momentum': 0.0003823158016201727, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:10:35,634] Trial 213 finished with value: 0.42699718475341797 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0021634930775427494, 'momentum': 0.00038143750988858636, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:10:35,847] Trial 214 finished with value: 0.4377245306968689 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009930691956372834, 'epsilon': 3.7998127563281513e-06, 'beta_1': 0.950583848149651, 'beta_2': 0.949356488305022}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:11:08,173] Trial 215 finished with value: 0.4275677800178528 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00041224361060755813, 'epsilon': 4.442339448845479e-06, 'beta_1': 0.9440378700851625, 'beta_2': 0.9495498737658601}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:11:15,436] Trial 216 finished with value: 0.4589191973209381 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00034243607715012667, 'epsilon': 3.6175329909419787e-06, 'beta_1': 0.9436586913609053, 'beta_2': 0.9475873780391064}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:11:17,897] Trial 218 finished with value: 0.4367726445198059 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042156115582835666, 'epsilon': 1.3946890818988888e-07, 'beta_1': 0.9267840448975976, 'beta_2': 0.9264912336163578}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:11:17,915] Trial 217 finished with value: 0.4395819306373596 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001048641767389483, 'epsilon': 4.30931115952755e-06, 'beta_1': 0.9439484622345993, 'beta_2': 0.9308478338728101}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:11:41,304] Trial 219 finished with value: 1.7738330364227295 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.01481962902952071, 'epsilon': 6.972096304610687e-06, 'beta_1': 0.9066464940319475, 'beta_2': 0.8846553540412149}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:11:57,030] Trial 220 finished with value: 0.4657350182533264 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0011184962902168086, 'epsilon': 6.793530779024828e-06, 'beta_1': 0.924002133177764, 'beta_2': 0.9287746952454542}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:00,358] Trial 221 finished with value: 0.5578833818435669 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0011556327174751084, 'momentum': 0.0007728755447050282, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:00,398] Trial 222 finished with value: 3.23724102973938 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0006039410885916105, 'momentum': 0.0005392519772042031, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:13,598] Trial 223 finished with value: 0.43119627237319946 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003066698211455936, 'momentum': 0.0008718147364175895, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:37,101] Trial 224 finished with value: 0.4312797486782074 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0032983679317807324, 'momentum': 0.02161750616657148, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:41,661] Trial 225 finished with value: 0.43941181898117065 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003017258610492675, 'momentum': 0.01725716636917261, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:42,401] Trial 226 finished with value: 0.4471692442893982 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0028488826021051752, 'momentum': 0.008297897724785952, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:12:50,062] Trial 227 finished with value: 0.4242321848869324 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003874491793976127, 'momentum': 0.021271132182769513, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:13:13,706] Trial 228 finished with value: 0.4068757891654968 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003689366554810647, 'momentum': 0.0002138458536364484, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:13:23,724] Trial 230 finished with value: 0.42649364471435547 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.004027125456378424, 'momentum': 0.00011934973393070852, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:13:24,671] Trial 229 finished with value: 0.4389558732509613 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.004192562388362901, 'momentum': 0.00022820736016104575, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:13:27,530] Trial 231 finished with value: 0.4356001317501068 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005999206457241057, 'momentum': 0.0001189728122651601, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:13:44,409] Trial 232 finished with value: 0.44047626852989197 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0068422655970738515, 'momentum': 0.0001966137133070918, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:06,935] Trial 235 finished with value: 0.42436546087265015 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.007909098130863776, 'momentum': 0.00019440055512678578, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:08,247] Trial 233 finished with value: 0.4331882894039154 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.006102548672365988, 'momentum': 0.00019568921111458025, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:08,826] Trial 234 finished with value: 0.4185400903224945 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.005949737015593052, 'momentum': 0.00033113272182570325, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:16,790] Trial 236 finished with value: 0.4271445870399475 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0025034022250391788, 'momentum': 0.00015547921878746515, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:49,497] Trial 237 finished with value: 0.4518423080444336 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.002321192314196196, 'momentum': 0.05433972626916917, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:50,501] Trial 238 finished with value: 0.4340762197971344 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.011192009403203683, 'momentum': 0.05702304213318429, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:50,543] Trial 239 finished with value: 0.43246617913246155 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.004941357939390668, 'momentum': 0.0003362410982470552, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:14:53,886] Trial 240 finished with value: 0.4844929277896881 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.004906955946745725, 'epsilon': 1.3944602312080112e-06, 'beta_1': 0.9374746816526944, 'beta_2': 0.9406207497415139}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:15:31,633] Trial 241 finished with value: 0.6377090215682983 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.010767126084824778, 'epsilon': 2.7046869203466928e-05, 'beta_1': 0.853471015462317, 'beta_2': 0.9182982182117888}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:15:33,706] Trial 244 finished with value: 0.4410015940666199 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.008520663976102284, 'momentum': 0.0278771726568528, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:15:33,777] Trial 243 finished with value: 0.42035746574401855 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.007901435563840885, 'epsilon': 0.001563525361738094, 'beta_1': 0.8501911762668537, 'beta_2': 0.9189774430845545}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:15:33,793] Trial 242 finished with value: 0.5052521228790283 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.005366853486042593, 'epsilon': 0.0018264820309132508, 'beta_1': 0.9369402498519454, 'beta_2': 0.8919364267277414}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:12,351] Trial 245 finished with value: 0.435441792011261 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0036857421948945533, 'momentum': 0.0004960863434884805, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:15,471] Trial 246 finished with value: 0.4491152763366699 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003529107273827551, 'momentum': 0.0002717715441443729, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:15,567] Trial 248 finished with value: 0.6973680257797241 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.007689440315142064, 'epsilon': 0.007366721415173588, 'beta_1': 0.8641057447790862, 'beta_2': 0.9064681479005643}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:15,616] Trial 247 finished with value: 0.5985264182090759 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.007636064094249692, 'epsilon': 0.004829264205613989, 'beta_1': 0.8637655804902244, 'beta_2': 0.9131544011894241}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:52,118] Trial 249 finished with value: 0.49294930696487427 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.007841046950609127, 'epsilon': 0.006089512541445253, 'beta_1': 0.8646164675923582, 'beta_2': 0.9124689373407766}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:56,155] Trial 252 finished with value: 0.8438133001327515 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.009749108824892638, 'epsilon': 0.0008682338268200322, 'beta_1': 0.8555963099179483, 'beta_2': 0.9237769280938333}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:56,177] Trial 251 finished with value: 1.8450708389282227 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.010082038208192519, 'epsilon': 1.8810136213722895e-08, 'beta_1': 0.8506198950009973, 'beta_2': 0.9249761553847888}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:16:56,217] Trial 250 finished with value: 0.44457054138183594 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0066000914670877025, 'epsilon': 1.3082176611355467e-05, 'beta_1': 0.94866670470659, 'beta_2': 0.9131368841141786}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:17:25,347] Trial 253 finished with value: 0.6183288097381592 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.009733482707259834, 'epsilon': 4.832192490677031e-07, 'beta_1': 0.9182829859443685, 'beta_2': 0.92438719899703}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:17:35,840] Trial 256 finished with value: 0.42440104484558105 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005461457500989163, 'epsilon': 1.1637338466045744e-06, 'beta_1': 0.9287173194303578, 'beta_2': 0.9331285191654345}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:17:36,445] Trial 254 finished with value: 0.4137154221534729 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005463137853341577, 'epsilon': 1.4284885392730499e-05, 'beta_1': 0.9485839815682868, 'beta_2': 0.9167597308753469}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:17:36,765] Trial 255 finished with value: 0.41648077964782715 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007014001287188587, 'epsilon': 1.5639635726732226e-05, 'beta_1': 0.9204240314387395, 'beta_2': 0.9338128760362299}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:17:53,356] Trial 257 finished with value: 0.43393969535827637 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.014215897776858695, 'momentum': 0.00031108235597393237, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:18:17,989] Trial 260 finished with value: 1.645875334739685 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0007888337601999614, 'momentum': 0.00032799730580190475, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:18:17,990] Trial 259 finished with value: 1.967284917831421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0007105050766101196, 'momentum': 0.00029202561988783644, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:18:17,993] Trial 258 finished with value: 0.4363355040550232 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.01738142031265914, 'momentum': 0.08876806342511342, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:18:26,915] Trial 261 finished with value: 2.191455364227295 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0007106615912397449, 'momentum': 8.160134597851235e-05, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:00,056] Trial 264 finished with value: 3.082690715789795 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005941088301826284, 'momentum': 0.0004273745530891904, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:00,335] Trial 263 finished with value: 2.8755576610565186 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005775471524320914, 'momentum': 8.762088994150241e-05, 'nesterov': False}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:01,343] Trial 262 finished with value: 0.8438655138015747 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0008924336298317229, 'momentum': 0.0663985698227279, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:05,344] Trial 265 finished with value: 0.45156678557395935 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009054473674088302, 'epsilon': 4.751154652581949e-08, 'beta_1': 0.9531505388009202, 'beta_2': 0.8788247910586247}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:44,612] Trial 267 finished with value: 0.43688473105430603 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004999596706823846, 'epsilon': 2.339162987684067e-05, 'beta_1': 0.9211993013105413, 'beta_2': 0.8948529660665864}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:46,102] Trial 268 finished with value: 0.40707531571388245 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046997291886001654, 'epsilon': 1.5158138687440812e-05, 'beta_1': 0.9664678996390631, 'beta_2': 0.8996761443000628}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:46,217] Trial 266 finished with value: 0.4331452250480652 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008577223264582595, 'epsilon': 2.3680264840221636e-05, 'beta_1': 0.9539607344827417, 'beta_2': 0.8772193022441699}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:19:46,297] Trial 269 finished with value: 0.4838084280490875 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0018234282788317528, 'epsilon': 2.4362572547272137e-05, 'beta_1': 0.9214489386147854, 'beta_2': 0.8946886585882166}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:20:25,724] Trial 270 finished with value: 0.5196478962898254 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0019391424547965054, 'rho': 0.9365990997759321, 'momentum': 0.04556491418137992, 'epsilon': 4.19209670413734e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:20:28,032] Trial 271 finished with value: 4.406809329986572 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0003652078223421851, 'momentum': 0.045816965423019954, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:20:28,265] Trial 272 finished with value: 0.41447940468788147 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003301401219830538, 'rho': 0.9343852558437848, 'momentum': 0.04984482903345002, 'epsilon': 9.208725572334845e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:20:28,424] Trial 273 finished with value: 0.44960325956344604 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00029107864690324317, 'rho': 0.934915509635241, 'momentum': 0.044676980845251846, 'epsilon': 1.01611335584147e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:05,861] Trial 274 finished with value: 4.244633197784424 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0003810510176604276, 'momentum': 0.0006216597162227736, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:06,359] Trial 275 finished with value: 4.491551399230957 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0002950928088188132, 'momentum': 0.02976923370974204, 'nesterov': True}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:08,369] Trial 276 finished with value: 0.442523330450058 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00029550988480581754, 'rho': 0.9582104384114436, 'momentum': 0.031084890247458394, 'epsilon': 1.6327041128423483e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:08,484] Trial 277 finished with value: 0.4252256453037262 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003227651155951733, 'rho': 0.9592893391606204, 'momentum': 0.029618858810888056, 'epsilon': 1.741729460220028e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:45,938] Trial 278 finished with value: 0.4506804943084717 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00029146121228297735, 'rho': 0.9592045754088867, 'momentum': 0.08122777531328766, 'epsilon': 1.405426469667743e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:47,662] Trial 279 finished with value: 0.41106128692626953 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00042161052160905185, 'rho': 0.959731555153007, 'momentum': 0.06644221075592116, 'epsilon': 1.5870035730594094e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:47,853] Trial 280 finished with value: 0.4238954484462738 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003823920091663185, 'rho': 0.9188624168746653, 'momentum': 0.06681346549574621, 'epsilon': 1.566310969427664e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:21:47,915] Trial 281 finished with value: 0.46392983198165894 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004754478430802199, 'rho': 0.9196804918223171, 'momentum': 0.06834320968252519, 'epsilon': 9.756007443373523e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:22:25,489] Trial 282 finished with value: 0.406533420085907 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000500467595749985, 'rho': 0.8639271687679726, 'momentum': 0.0668538524588851, 'epsilon': 4.765429013503705e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:22:28,172] Trial 283 finished with value: 0.4265323579311371 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004267262100229609, 'rho': 0.9646677308188836, 'momentum': 0.09962002356237319, 'epsilon': 9.206214046667433e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:22:28,269] Trial 285 finished with value: 0.4273358881473541 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00022081223218736002, 'rho': 0.9529903994759105, 'momentum': 0.09622992545058841, 'epsilon': 7.239349285744503e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:22:28,371] Trial 284 finished with value: 0.43829822540283203 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00046719218366522366, 'rho': 0.9679202285474645, 'momentum': 0.056560462039088925, 'epsilon': 9.053081612211543e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:04,482] Trial 286 finished with value: 5.2577314376831055 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 1.0503839475424636e-05, 'rho': 0.9657566726098478, 'momentum': 0.09237769537014774, 'epsilon': 5.047636417826442e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:08,854] Trial 289 finished with value: 0.4183402955532074 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00040362854108161187, 'rho': 0.9781385712734484, 'momentum': 0.00014002220319021745, 'epsilon': 7.744154039724837e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:09,163] Trial 287 finished with value: 0.40689390897750854 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004944607935425622, 'rho': 0.9804130413138759, 'momentum': 0.04873430608668307, 'epsilon': 4.449579278316369e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:09,292] Trial 288 finished with value: 0.4323273301124573 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00046926124878796755, 'rho': 0.9402860864224858, 'momentum': 0.05401497990183788, 'epsilon': 5.743387520661719e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:40,828] Trial 290 finished with value: 0.44187307357788086 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005355021725939358, 'rho': 0.9408432789058998, 'momentum': 0.07859946277119459, 'epsilon': 1.2083644914487406e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:50,091] Trial 292 finished with value: 0.4232181906700134 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000494919766017995, 'rho': 0.9838404579756818, 'momentum': 0.05092142872088379, 'epsilon': 7.318779542916139e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:51,183] Trial 291 finished with value: 0.41656604409217834 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005200936233641539, 'rho': 0.9827866729616478, 'momentum': 0.0001375190176555971, 'epsilon': 6.724579247542252e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:23:51,186] Trial 293 finished with value: 0.4544512629508972 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00035474004431936576, 'rho': 0.9814397948797938, 'momentum': 6.074931950631853e-05, 'epsilon': 6.874789590845328e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:24:12,316] Trial 294 finished with value: 0.42334040999412537 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003494968530970499, 'rho': 0.982331181476638, 'momentum': 0.0001425556532456129, 'epsilon': 2.391952286038091e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:24:31,028] Trial 295 finished with value: 0.4347880482673645 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000654583989603774, 'rho': 0.9797101338214738, 'momentum': 0.037472107544370456, 'epsilon': 4.194455484304703e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:24:32,683] Trial 297 finished with value: 0.42919474840164185 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006241885788044446, 'rho': 0.9857634492876423, 'momentum': 0.037839959724657486, 'epsilon': 2.122929338037078e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:24:32,850] Trial 296 finished with value: 0.44171467423439026 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005576626179608998, 'rho': 0.9895534952483523, 'momentum': 0.03761004333027111, 'epsilon': 2.9955628545989865e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:24:44,553] Trial 298 finished with value: 0.45524460077285767 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005919636902710235, 'rho': 0.9891778778075703, 'momentum': 0.034583614196693264, 'epsilon': 4.086877119564809e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:12,554] Trial 299 finished with value: 0.43534785509109497 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005964832100839666, 'rho': 0.9725104458572146, 'momentum': 0.0001687668656147446, 'epsilon': 4.0099163377992826e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:15,087] Trial 300 finished with value: 0.44126278162002563 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005488708397864699, 'rho': 0.9886228871795699, 'momentum': 0.07316082670957014, 'epsilon': 3.9773017455522494e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:15,116] Trial 301 finished with value: 0.4089585244655609 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00040264452791448525, 'rho': 0.8568998810274243, 'momentum': 0.07173935140786648, 'epsilon': 4.077214592050111e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:21,331] Trial 302 finished with value: 0.4391799569129944 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000436202084647466, 'rho': 0.8614335060636285, 'momentum': 0.07405431921514205, 'epsilon': 3.0726944718113746e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:55,153] Trial 303 finished with value: 0.42796483635902405 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004455565635029908, 'rho': 0.9310148021342725, 'momentum': 0.07299279327923583, 'epsilon': 3.00582721051979e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:59,145] Trial 305 finished with value: 0.4387229084968567 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00042997146099314434, 'rho': 0.8613795480610439, 'momentum': 0.06306148138924131, 'epsilon': 6.948305821859093e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:25:59,192] Trial 304 finished with value: 0.44159436225891113 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00046034247243795657, 'rho': 0.8592894506926742, 'momentum': 0.06045273890562421, 'epsilon': 8.638749594806155e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:26:00,734] Trial 306 finished with value: 0.4376567006111145 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004258227091824185, 'rho': 0.856940089718046, 'momentum': 0.06033038352319582, 'epsilon': 8.578735728252146e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:26:33,675] Trial 307 finished with value: 0.41815775632858276 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002526399692358783, 'rho': 0.8514497395379887, 'momentum': 0.05957882508572126, 'epsilon': 1.4646932951183023e-08}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:26:40,578] Trial 309 finished with value: 0.42202824354171753 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00039700272867329117, 'rho': 0.8507620692426651, 'momentum': 0.00011398020305628755, 'epsilon': 1.0261463402344245e-09}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:26:41,125] Trial 308 finished with value: 0.4450700879096985 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002555059226633626, 'rho': 0.8678576792787603, 'momentum': 0.00010723165863135515, 'epsilon': 0.00013204741911733772}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:26:43,024] Trial 310 finished with value: 0.4483865797519684 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003609077149218548, 'rho': 0.8531777542852936, 'momentum': 0.0001135870663062374, 'epsilon': 0.00015275451255136312}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:27:05,965] Trial 311 finished with value: 0.4409317970275879 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003402401537899671, 'rho': 0.8676774728734413, 'momentum': 0.04607794510803991, 'epsilon': 1.1975280195475928e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:27:22,337] Trial 312 finished with value: 0.45200276374816895 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00033869695032666424, 'rho': 0.8662806046461535, 'momentum': 9.076391962429598e-05, 'epsilon': 1.3302957414482264e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:27:23,737] Trial 313 finished with value: 5.537788391113281 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 1.9334419717121113e-05, 'rho': 0.8855466364457988}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:27:24,804] Trial 314 finished with value: 5.271078586578369 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.002674330197689823, 'rho': 0.8942302073252788}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:27:37,954] Trial 315 finished with value: 5.6085524559021 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0006592820731994991, 'rho': 0.8881603120826992}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:05,253] Trial 316 finished with value: 5.250239372253418 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.002622853816514061, 'rho': 0.8885239111691906}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:05,339] Trial 317 finished with value: 0.44159555435180664 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.002560172677708083, 'epsilon': 2.943039638551215e-05, 'beta_1': 0.9677604017268987, 'beta_2': 0.899494227519432}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:07,034] Trial 318 finished with value: 0.4497753977775574 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007114251322435173, 'epsilon': 5.259321926724471e-05, 'beta_1': 0.9588494846871849, 'beta_2': 0.8995093269624455}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:12,767] Trial 319 finished with value: 0.43726474046707153 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0015787503081637832, 'epsilon': 5.037066483031026e-06, 'beta_1': 0.966932999138662, 'beta_2': 0.9828781247558543}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:47,257] Trial 320 finished with value: 0.43018466234207153 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005250464334418032, 'epsilon': 2.9527453466200747e-05, 'beta_1': 0.9661523553445042, 'beta_2': 0.9048247783864386}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:49,839] Trial 322 finished with value: 0.46193379163742065 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001330788498447499, 'epsilon': 2.2572532131174577e-05, 'beta_1': 0.9725131172101563, 'beta_2': 0.8856314434732725}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:50,015] Trial 321 finished with value: 0.42808961868286133 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005138508263925165, 'epsilon': 5.445834767426561e-06, 'beta_1': 0.9589265571755592, 'beta_2': 0.904133084432214}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:28:52,744] Trial 323 finished with value: 0.4297601580619812 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005139780846475332, 'epsilon': 1.9400102936365826e-05, 'beta_1': 0.9726524833125334, 'beta_2': 0.9051169762746707}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:29:27,014] Trial 324 finished with value: 0.43827858567237854 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005076397120328135, 'epsilon': 1.7627002688527077e-05, 'beta_1': 0.9767788206554372, 'beta_2': 0.8898002664973728}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:29:32,672] Trial 326 finished with value: 0.6560629606246948 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.004573190676534649, 'rho': 0.9028662095707825, 'momentum': 0.00022620434502925726, 'epsilon': 1.761523590536061e-05}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:29:32,703] Trial 325 finished with value: 0.41313090920448303 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000495558076493052, 'epsilon': 7.075986383424424e-06, 'beta_1': 0.9773622772166, 'beta_2': 0.9026494447336969}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:29:32,734] Trial 327 finished with value: 0.4576416611671448 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00020178837172443286, 'rho': 0.9509091113537049, 'momentum': 0.08682960495252631, 'epsilon': 7.94158962973015e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:03,078] Trial 328 finished with value: 0.5257421135902405 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.003233831378517958, 'rho': 0.8782815421527376, 'momentum': 0.0002190986461720068, 'epsilon': 7.5775808771886834e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:14,483] Trial 330 finished with value: 0.45693621039390564 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006443726532897871, 'rho': 0.9717397426015261, 'momentum': 0.02355203828592688, 'epsilon': 3.1738276908566835e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:15,815] Trial 329 finished with value: 0.5157595872879028 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002064680246533397, 'rho': 0.8805891414989523, 'momentum': 0.09653248332314227, 'epsilon': 3.1541490672555594e-06}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:16,609] Trial 331 finished with value: 0.4373388886451721 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006621691599919206, 'rho': 0.8751829413529565, 'momentum': 0.050248747710313305, 'epsilon': 1.1253172681450675e-08}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:36,870] Trial 332 finished with value: 0.4639872908592224 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000617405534527979, 'epsilon': 1.500096626211447e-09, 'beta_1': 0.9876318581675422, 'beta_2': 0.9161788620258531}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:54,839] Trial 333 finished with value: 0.42704591155052185 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003142865172484538, 'epsilon': 2.1591126430990873e-08, 'beta_1': 0.9638151252650621, 'beta_2': 0.8536592238340475}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:56,557] Trial 334 finished with value: 0.4321897625923157 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004103632530174074, 'epsilon': 1.5051385123084214e-09, 'beta_1': 0.9637247212223519, 'beta_2': 0.9086546344643748}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:30:57,539] Trial 335 finished with value: 0.42865973711013794 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00038959551877335355, 'epsilon': 5.2684910510164476e-05, 'beta_1': 0.9859297830578784, 'beta_2': 0.9098786459957761}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:31:10,843] Trial 336 finished with value: 0.40699100494384766 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003999221992390987, 'epsilon': 5.4690929462061136e-05, 'beta_1': 0.9612282608844611, 'beta_2': 0.8526921452286336}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:31:35,344] Trial 337 finished with value: 0.45499691367149353 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004079319871650492, 'epsilon': 1.7480629827685636e-06, 'beta_1': 0.9760507230395382, 'beta_2': 0.8704736310674385}. Best is trial 69 with value: 0.4006522595882416. [I 2021-03-10 20:31:37,380] Trial 338 finished with value: 0.3914160132408142 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00038881920559352023, 'epsilon': 5.417737789477851e-05, 'beta_1': 0.9851634781172875, 'beta_2': 0.870855712415811}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:31:38,096] Trial 339 finished with value: 0.4542507529258728 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007392670036300359, 'epsilon': 1.1880177139617063e-05, 'beta_1': 0.9477272730027541, 'beta_2': 0.8833619437909572}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:31:44,396] Trial 340 finished with value: 0.45723652839660645 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00027576280711377716, 'epsilon': 1.1028423100008373e-05, 'beta_1': 0.9829469834259674, 'beta_2': 0.8549805850563456}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:32:17,735] Trial 341 finished with value: 0.4167390763759613 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028943677917630275, 'rho': 0.9621577582234444, 'momentum': 0.09855834009511535, 'epsilon': 6.522448939432053e-05}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:32:20,406] Trial 342 finished with value: 0.43101829290390015 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003094422375859294, 'epsilon': 6.406532069053384e-05, 'beta_1': 0.9839336348000512, 'beta_2': 0.861123514461081}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:32:20,775] Trial 343 finished with value: 0.4195035398006439 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002735428444778577, 'epsilon': 6.147572708146096e-05, 'beta_1': 0.9823045941374704, 'beta_2': 0.8603778989326165}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:32:23,118] Trial 344 finished with value: 0.4337705075740814 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003244149262698409, 'epsilon': 6.586225848272219e-05, 'beta_1': 0.9776296032153585, 'beta_2': 0.8649197928153827}. Best is trial 338 with value: 0.3914160132408142. [I 2021-03-10 20:32:58,396] Trial 345 finished with value: 0.38822034001350403 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035718724976417535, 'epsilon': 3.5043808140212366e-05, 'beta_1': 0.9834335585011938, 'beta_2': 0.8583875028859446}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:02,281] Trial 346 finished with value: 0.44426679611206055 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00040024174644897925, 'epsilon': 4.088606594294759e-05, 'beta_1': 0.9792366706985544, 'beta_2': 0.9012745120310187}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:02,807] Trial 347 finished with value: 0.414922297000885 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035997751256248295, 'epsilon': 0.00010403601820377798, 'beta_1': 0.979223343325224, 'beta_2': 0.8642114860153183}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:03,322] Trial 348 finished with value: 0.4104757308959961 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003610255906328136, 'epsilon': 3.293113210948469e-05, 'beta_1': 0.974315653391454, 'beta_2': 0.8583164851920583}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:38,400] Trial 349 finished with value: 0.42523032426834106 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003871276139661485, 'epsilon': 4.36287677088783e-05, 'beta_1': 0.973040622393027, 'beta_2': 0.8580336722962923}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:44,929] Trial 350 finished with value: 0.41518259048461914 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035445536124585473, 'epsilon': 9.877407369598038e-05, 'beta_1': 0.988906211967037, 'beta_2': 0.8577912039723984}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:46,221] Trial 352 finished with value: 0.4209307134151459 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003439880114116538, 'epsilon': 0.00011058311735844965, 'beta_1': 0.9735794726219053, 'beta_2': 0.8587118071288398}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:33:46,257] Trial 351 finished with value: 0.4311304986476898 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00033918638979824284, 'epsilon': 0.00014760925849171367, 'beta_1': 0.9889555954903337, 'beta_2': 0.8584631936538305}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:34:13,526] Trial 353 finished with value: 0.47731247544288635 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00022345705340415164, 'epsilon': 9.507518506572696e-05, 'beta_1': 0.9738747718280278, 'beta_2': 0.8504894623424353}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:34:25,825] Trial 354 finished with value: 0.42303138971328735 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032968935188987497, 'epsilon': 0.00019455009621597918, 'beta_1': 0.9894661134183786, 'beta_2': 0.8520623909336049}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:34:27,559] Trial 356 finished with value: 0.5251965522766113 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00016679848327770438, 'epsilon': 0.00017493056001832144, 'beta_1': 0.969654358990779, 'beta_2': 0.851053787301245}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:34:28,057] Trial 355 finished with value: 0.4685315489768982 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00023867728541016224, 'epsilon': 2.8919560671018112e-05, 'beta_1': 0.9861060574517271, 'beta_2': 0.8650461475401279}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:34:45,199] Trial 357 finished with value: 0.48209238052368164 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026597035971054025, 'epsilon': 0.00018172809290442488, 'beta_1': 0.989829171020246, 'beta_2': 0.8649217801129908}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:06,598] Trial 358 finished with value: 0.5010387897491455 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00028390115117668134, 'epsilon': 3.168175997521412e-05, 'beta_1': 0.9852494268879036, 'beta_2': 0.8668463708856121}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:08,082] Trial 359 finished with value: 0.45124000310897827 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004468696632119843, 'epsilon': 3.0247630070151655e-05, 'beta_1': 0.9845194222919245, 'beta_2': 0.8644549911075374}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:08,457] Trial 360 finished with value: 0.4681217670440674 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026603558583474006, 'epsilon': 3.203528742893481e-05, 'beta_1': 0.9789882923605299, 'beta_2': 0.8711795828845936}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:18,643] Trial 361 finished with value: 0.4270513653755188 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00043421308937681327, 'epsilon': 3.332413315271859e-05, 'beta_1': 0.9797312855741243, 'beta_2': 0.8672769473184178}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:47,440] Trial 362 finished with value: 0.4418056011199951 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00044580910961012824, 'epsilon': 8.976429055630885e-05, 'beta_1': 0.9791109324279251, 'beta_2': 0.8698257193238483}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:49,880] Trial 363 finished with value: 0.43377095460891724 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000391651266174898, 'epsilon': 9.084346817334113e-05, 'beta_1': 0.9793832485836951, 'beta_2': 0.869181556571232}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:49,922] Trial 364 finished with value: 0.4353521764278412 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003625430819878852, 'epsilon': 8.198153102269084e-05, 'beta_1': 0.9809574128360959, 'beta_2': 0.8558316666909882}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:35:52,768] Trial 365 finished with value: 0.4226773679256439 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00037562141891641347, 'epsilon': 0.0001057594394178432, 'beta_1': 0.9616506736981841, 'beta_2': 0.8740405674272007}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:36:26,991] Trial 366 finished with value: 0.4107280671596527 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035930014299974275, 'epsilon': 0.0003233750985594027, 'beta_1': 0.9606053852896609, 'beta_2': 0.8550881194225772}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:36:30,602] Trial 367 finished with value: 0.44232767820358276 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003414518470644648, 'epsilon': 2.1430809402941192e-05, 'beta_1': 0.9603538740741449, 'beta_2': 0.8607274844111324}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:36:30,624] Trial 368 finished with value: 0.44866475462913513 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032990544805122196, 'epsilon': 2.160794970179059e-05, 'beta_1': 0.9694590321296674, 'beta_2': 0.8603013975124447}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:36:30,793] Trial 369 finished with value: 0.4317162036895752 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00033468826785874607, 'rho': 0.9145956568073015, 'momentum': 0.015517261367768624, 'epsilon': 2.218407770286151e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:04,684] Trial 370 finished with value: 0.4195161759853363 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032311404625564086, 'epsilon': 4.639678527323653e-05, 'beta_1': 0.9654722378028628, 'beta_2': 0.8547102606749682}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:10,150] Trial 371 finished with value: 0.41724881529808044 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00030962674664826743, 'epsilon': 7.078370568623662e-06, 'beta_1': 0.9673976577485839, 'beta_2': 0.854353230666075}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:10,605] Trial 373 finished with value: 0.450577974319458 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004886257097391386, 'epsilon': 5.1110108232660956e-05, 'beta_1': 0.975790571245908, 'beta_2': 0.8533975901221021}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:10,736] Trial 372 finished with value: 0.4054027199745178 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00047907429399704385, 'epsilon': 0.0004079515459807176, 'beta_1': 0.9657194638269309, 'beta_2': 0.8547019541357405}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:35,287] Trial 374 finished with value: 0.4885069727897644 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001793286757668317, 'epsilon': 0.00042893639700656867, 'beta_1': 0.9565984845092963, 'beta_2': 0.8501508242984148}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:50,632] Trial 375 finished with value: 0.43943849205970764 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005059497277001694, 'epsilon': 0.0007504707000800808, 'beta_1': 0.9756372923893987, 'beta_2': 0.8630800206556087}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:51,564] Trial 377 finished with value: 0.438521146774292 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004620683758116366, 'epsilon': 0.0004262218789464429, 'beta_1': 0.9551486508671939, 'beta_2': 0.8509137812538713}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:37:52,091] Trial 376 finished with value: 0.42589783668518066 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00025799306447276023, 'epsilon': 0.0003521341488277704, 'beta_1': 0.9566162176859895, 'beta_2': 0.8503723755551704}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:38:03,931] Trial 378 finished with value: 0.43853873014450073 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004379860954181415, 'rho': 0.9551267019462383, 'momentum': 0.01926953794014188, 'epsilon': 0.0006095998065409554}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:38:33,023] Trial 379 finished with value: 0.4125490188598633 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00043448274800368203, 'rho': 0.9549666221215645, 'momentum': 0.003285799862489389, 'epsilon': 4.870239070637564e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:38:33,728] Trial 381 finished with value: 0.42942649126052856 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00043091140736632597, 'rho': 0.9466442361312603, 'momentum': 0.006855909702915682, 'epsilon': 0.0007446727885347604}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:38:33,755] Trial 380 finished with value: 0.43122395873069763 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00043907509416441317, 'rho': 0.9416124001238332, 'momentum': 0.025548236313303815, 'epsilon': 0.00023747587553042145}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:38:38,226] Trial 382 finished with value: 0.52265465259552 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023172350520743614, 'rho': 0.94529511945661, 'momentum': 0.0039177578832963736, 'epsilon': 0.00023958256940728186}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:39:16,245] Trial 383 finished with value: 0.41512614488601685 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005078657021997323, 'rho': 0.9423018437987343, 'momentum': 0.0013084379775708088, 'epsilon': 4.1398608880442485e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:39:17,480] Trial 385 finished with value: 0.44809651374816895 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005346178091264353, 'rho': 0.9238164005627458, 'momentum': 0.0035196691977363564, 'epsilon': 5.563707502357598e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:39:17,591] Trial 384 finished with value: 0.4403596520423889 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005611522670914851, 'rho': 0.9268628373937982, 'momentum': 0.0013607557064574077, 'epsilon': 4.447368063903706e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:39:18,696] Trial 386 finished with value: 0.39942044019699097 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005278763450887161, 'rho': 0.9269643554540786, 'momentum': 0.004179134617825752, 'epsilon': 2.3244042905535494e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:39:59,055] Trial 387 finished with value: 0.5878629684448242 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 8.182921361460663e-05, 'rho': 0.9220149563764344, 'momentum': 0.00220667934540669, 'epsilon': 5.6591714244652986e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:00,730] Trial 388 finished with value: 0.4151056408882141 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005485178306458691, 'rho': 0.9325456200078734, 'momentum': 0.0023823074542552213, 'epsilon': 0.0010950206442510767}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:01,571] Trial 390 finished with value: 0.4215336740016937 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000590207785230806, 'rho': 0.9343570342475626, 'momentum': 0.004903162006157437, 'epsilon': 2.294106662546036e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:02,017] Trial 389 finished with value: 0.6844775676727295 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 9.383928138238136e-05, 'rho': 0.9317037032792932, 'momentum': 0.050850088162588215, 'epsilon': 2.6771463865321723e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:42,184] Trial 391 finished with value: 0.4299391806125641 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005361574241639483, 'rho': 0.9330887543773241, 'momentum': 0.0041389457480979375, 'epsilon': 2.5744820340871922e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:43,672] Trial 392 finished with value: 0.4133198857307434 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004125860161116073, 'rho': 0.9688551296629749, 'momentum': 0.0043700123904328045, 'epsilon': 1.999179803616203e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:45,225] Trial 393 finished with value: 0.4276120066642761 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00039122554342012155, 'rho': 0.9373366860022719, 'momentum': 0.0020396315399377903, 'epsilon': 2.8270389860908764e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:40:45,227] Trial 394 finished with value: 0.46909427642822266 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004103347412826158, 'rho': 0.9497981829841191, 'momentum': 0.003127178273914867, 'epsilon': 1.1276168513234257e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:41:25,059] Trial 395 finished with value: 0.41885313391685486 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003868007269591599, 'rho': 0.9751297303331784, 'momentum': 0.0029180790296251683, 'epsilon': 1.0799124261133475e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:41:28,101] Trial 396 finished with value: 0.41341835260391235 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00042648242302110046, 'rho': 0.9677386103199814, 'momentum': 0.004577622122944997, 'epsilon': 1.096211582440457e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:41:28,383] Trial 398 finished with value: 0.45529359579086304 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004835343468650698, 'rho': 0.9672562298010867, 'momentum': 0.0059721373212497765, 'epsilon': 1.7231039886107719e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:41:28,391] Trial 397 finished with value: 0.4198642373085022 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004604738735282629, 'rho': 0.9679529913342059, 'momentum': 0.008377995056302445, 'epsilon': 1.2394576030894179e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:07,058] Trial 399 finished with value: 0.40722545981407166 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028451869814982397, 'rho': 0.9668063496100011, 'momentum': 0.006332007288954075, 'epsilon': 1.22898548353554e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:11,914] Trial 402 finished with value: 0.463955819606781 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00039789574940670363, 'rho': 0.9642677974898596, 'momentum': 0.0050109661898413505, 'epsilon': 7.921237893172981e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:12,191] Trial 401 finished with value: 0.443431556224823 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004519024770950628, 'rho': 0.9613173747906913, 'momentum': 0.004613903944140132, 'epsilon': 1.6017250010642884e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:12,228] Trial 400 finished with value: 0.43432170152664185 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004495325358279044, 'rho': 0.9676262582699849, 'momentum': 0.004755664934431295, 'epsilon': 1.291625912727139e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:43,451] Trial 403 finished with value: 0.4198470115661621 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003026226471907989, 'rho': 0.9614909516397736, 'momentum': 0.004758215584888617, 'epsilon': 7.228896069799825e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:52,671] Trial 406 finished with value: 0.43703946471214294 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028188739047917095, 'rho': 0.9776225077408144, 'momentum': 0.00614876746294213, 'epsilon': 1.7687680593750873e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:54,112] Trial 405 finished with value: 0.43953579664230347 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002977712189130377, 'rho': 0.9780741728413438, 'momentum': 0.006000678448816792, 'epsilon': 9.952298712860323e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:42:54,139] Trial 404 finished with value: 0.4514892101287842 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023524836851226104, 'rho': 0.9779629068726781, 'momentum': 0.004562318124964716, 'epsilon': 9.89367832387006e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:43:14,293] Trial 407 finished with value: 0.47066837549209595 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00021640718426358955, 'rho': 0.9790478506284435, 'momentum': 0.00615716607040571, 'epsilon': 1.035685052245065e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:43:34,607] Trial 408 finished with value: 0.5073796510696411 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00021869927981836912, 'rho': 0.9707344440875972, 'momentum': 0.001671636940237012, 'epsilon': 1.5797802057606955e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:43:36,176] Trial 410 finished with value: 0.3993247151374817 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005401805162011991, 'rho': 0.9572636423777527, 'momentum': 0.001649593640648112, 'epsilon': 1.4881888602535598e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:43:36,195] Trial 409 finished with value: 0.44963327050209045 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006114615941744591, 'rho': 0.9707906032112791, 'momentum': 0.0025616627520023937, 'epsilon': 2.152232797788591e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:43:46,451] Trial 411 finished with value: 0.4479503035545349 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000572074024484364, 'rho': 0.9566120189877055, 'momentum': 0.0016965378282608624, 'epsilon': 3.6938322027306725e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:44:15,976] Trial 412 finished with value: 0.4441708028316498 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000597643681309351, 'rho': 0.9864402262365257, 'momentum': 0.003545971864141099, 'epsilon': 2.0600662243270295e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:44:18,701] Trial 414 finished with value: 0.43657952547073364 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003780742002294095, 'rho': 0.9541870345464246, 'momentum': 0.0009890738279999742, 'epsilon': 5.391903796726782e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:44:18,726] Trial 413 finished with value: 0.4224017262458801 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00038023059321624887, 'rho': 0.9566918796766938, 'momentum': 0.0035160471187858757, 'epsilon': 1.6432456607771883e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:44:22,793] Trial 415 finished with value: 0.4357338547706604 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00037616018572223127, 'rho': 0.9737510089005881, 'momentum': 0.0073815397680585025, 'epsilon': 5.545402555744915e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:44:56,369] Trial 416 finished with value: 0.4403154253959656 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00037097013008573557, 'rho': 0.954819148488695, 'momentum': 0.0074103571446686445, 'epsilon': 1.4444639803754644e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:00,342] Trial 418 finished with value: 0.4545968174934387 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004702144660563829, 'rho': 0.9590768010510922, 'momentum': 0.0029655876621435943, 'epsilon': 1.369009402873464e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:01,666] Trial 417 finished with value: 0.4535399079322815 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003700327213694084, 'rho': 0.9596522644664078, 'momentum': 0.009970707867600717, 'epsilon': 7.423461388130947e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:02,803] Trial 419 finished with value: 0.422015905380249 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00048576196388583115, 'rho': 0.9643816026367696, 'momentum': 0.0029884218450353037, 'epsilon': 3.633112528958666e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:34,382] Trial 420 finished with value: 0.42520871758461 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028135020486831343, 'rho': 0.9591216129789628, 'momentum': 0.003406827725315249, 'epsilon': 3.0706531208584063e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:42,259] Trial 421 finished with value: 0.4434855878353119 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00027199978760684545, 'rho': 0.9520371522026652, 'momentum': 0.0007296988586132508, 'epsilon': 3.36499786457156e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:42,290] Trial 422 finished with value: 0.43013447523117065 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003013379649526917, 'rho': 0.9511946444973867, 'momentum': 0.001853552912180517, 'epsilon': 3.561184995504351e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:45:43,305] Trial 423 finished with value: 0.4503725469112396 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002955596908586641, 'rho': 0.9518666348830599, 'momentum': 0.0007910504247830797, 'epsilon': 3.038785070745679e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:46:06,690] Trial 424 finished with value: 0.4128482937812805 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002966700671337858, 'rho': 0.9520461066688266, 'momentum': 0.00404076483373834, 'epsilon': 2.0729833998863136e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:46:26,548] Trial 426 finished with value: 0.44487014412879944 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006836980154946348, 'rho': 0.9653948534464533, 'momentum': 0.0013617990058389432, 'epsilon': 4.065140933127666e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:46:26,570] Trial 425 finished with value: 0.4208546578884125 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004799439353963841, 'rho': 0.9741462753975638, 'momentum': 0.0023607507846657924, 'epsilon': 4.3291444072634235e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:46:26,579] Trial 427 finished with value: 0.4394358694553375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006925627051051409, 'rho': 0.9697920683276434, 'momentum': 0.0009609624559148072, 'epsilon': 4.203085584640479e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:46:37,419] Trial 428 finished with value: 0.42322486639022827 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00046874283164134847, 'rho': 0.965783057326684, 'momentum': 0.002383543383718372, 'epsilon': 2.4186081200163764e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:10,508] Trial 431 finished with value: 0.44683098793029785 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00018787897132750746, 'rho': 0.9860564087172459, 'momentum': 0.003951694326642869, 'epsilon': 2.3917149853465415e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:11,872] Trial 429 finished with value: 0.44855380058288574 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000248191124411231, 'rho': 0.9486555841166388, 'momentum': 0.004305823859119177, 'epsilon': 1.891755522438395e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:11,966] Trial 430 finished with value: 0.43209367990493774 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00019903274101584425, 'rho': 0.9819664241632151, 'momentum': 0.0036565386692479205, 'epsilon': 1.0691110681374163e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:16,076] Trial 432 finished with value: 0.4572431445121765 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002526625266064734, 'rho': 0.9810810528164086, 'momentum': 0.004097700435253615, 'epsilon': 1.0298724680720638e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:55,114] Trial 434 finished with value: 0.4071427285671234 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003347668133883041, 'rho': 0.8645783519309771, 'momentum': 0.0011445139854435208, 'epsilon': 3.641296091322728e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:55,116] Trial 435 finished with value: 0.46420150995254517 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003568605854804178, 'rho': 0.9011139753169112, 'momentum': 0.0011874589745014385, 'epsilon': 5.329110798785394e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:55,143] Trial 433 finished with value: 0.4773748517036438 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023559969918470278, 'rho': 0.9628988408355222, 'momentum': 0.002871202368242223, 'epsilon': 4.7029202086109355e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:47:56,864] Trial 436 finished with value: 0.4605432152748108 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003331994399092611, 'rho': 0.9627804419790115, 'momentum': 0.0011802017350810922, 'epsilon': 4.45917593468265e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:48:36,945] Trial 437 finished with value: 0.41002798080444336 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002895708641215061, 'rho': 0.8701472355502617, 'momentum': 0.0011068460709197242, 'epsilon': 3.880873402947865e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:48:38,255] Trial 440 finished with value: 0.4749080240726471 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003224675394530222, 'rho': 0.8635605696320667, 'momentum': 0.0006060330126807615, 'epsilon': 4.03224882552207e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:48:38,488] Trial 439 finished with value: 5.5209174156188965 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00015310596565877232, 'rho': 0.8554850725275194}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:48:38,559] Trial 438 finished with value: 5.630482196807861 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00032370824408215463, 'rho': 0.8700861128730027}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:49:18,585] Trial 441 finished with value: 5.461050987243652 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0003143411128202579, 'rho': 0.85562021641587}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:49:19,693] Trial 442 finished with value: 5.5136308670043945 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0002753951745125138, 'rho': 0.870313622922169}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:49:19,776] Trial 444 finished with value: 0.4431583881378174 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000279309598765657, 'rho': 0.8590770748937266, 'momentum': 0.0015651319313128396, 'epsilon': 3.1660113948550755e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:49:20,285] Trial 443 finished with value: 0.42345404624938965 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028406094819260135, 'rho': 0.8695405332421359, 'momentum': 0.0004864029124891089, 'epsilon': 2.5991702837620594e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:01,973] Trial 445 finished with value: 0.3986162543296814 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00025836539663363537, 'rho': 0.8645735402096537, 'momentum': 0.0010771053611559032, 'epsilon': 3.0323559991135264e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:03,714] Trial 446 finished with value: 0.48571038246154785 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00025282648158873904, 'rho': 0.8631942868093243, 'momentum': 0.001011139766245876, 'epsilon': 2.5794297870563756e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:04,017] Trial 448 finished with value: 0.44319677352905273 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005434899654115164, 'rho': 0.8648939964833425, 'momentum': 0.0009570114185447953, 'epsilon': 2.4342076939943672e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:04,480] Trial 447 finished with value: 0.48279842734336853 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000548965456345398, 'rho': 0.9434576039011892, 'momentum': 0.0009736625086163979, 'epsilon': 2.3839022647328342e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:44,210] Trial 449 finished with value: 0.5336755514144897 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00018077285367238828, 'rho': 0.8639310584689203, 'momentum': 0.0010013112953406018, 'epsilon': 2.3537991436227934e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:48,174] Trial 452 finished with value: 0.4493377208709717 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002103624257170917, 'rho': 0.8590781089933835, 'momentum': 0.0018780265693860347, 'epsilon': 6.584530536035971e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:48,194] Trial 451 finished with value: 0.46003589034080505 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023541842097577086, 'rho': 0.8762403063726161, 'momentum': 0.0018277427043577492, 'epsilon': 7.069662470985602e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:50:48,255] Trial 450 finished with value: 0.49217042326927185 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00018529156608196591, 'rho': 0.8650028469463722, 'momentum': 0.0016800018749095439, 'epsilon': 3.546388746476001e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:51:21,341] Trial 453 finished with value: 0.5182441473007202 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023119270907039744, 'rho': 0.8740655544860306, 'momentum': 0.0019738314652795685, 'epsilon': 6.191397298221982e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:51:28,839] Trial 454 finished with value: 0.44279757142066956 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003799252707073931, 'rho': 0.8536189675675346, 'momentum': 0.000742016005884319, 'epsilon': 3.782237447367022e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:51:29,644] Trial 456 finished with value: 0.42416009306907654 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003983029214547533, 'rho': 0.8501884467314637, 'momentum': 0.0014602133724158667, 'epsilon': 1.4712466539430084e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:51:29,646] Trial 455 finished with value: 0.42510923743247986 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003812757970273305, 'rho': 0.8512921273599512, 'momentum': 0.0007324746165032843, 'epsilon': 1.803702452968864e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:51:50,847] Trial 457 finished with value: 0.4467279016971588 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003789459369453831, 'rho': 0.8523466657649974, 'momentum': 0.0007327936093381796, 'epsilon': 0.0023519412664803102}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:09,696] Trial 458 finished with value: 0.44861865043640137 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00034706103016009986, 'rho': 0.8720168020842967, 'momentum': 0.0014173822107291386, 'epsilon': 1.6622360169685117e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:11,089] Trial 460 finished with value: 0.4336675703525543 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00031823257443266436, 'rho': 0.8581011555107869, 'momentum': 0.0011571384672462715, 'epsilon': 4.8758623907371536e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:11,143] Trial 459 finished with value: 0.5082861185073853 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00031901820976315987, 'rho': 0.8576052076621501, 'momentum': 0.00127764447931634, 'epsilon': 4.9333836002810994e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:22,207] Trial 461 finished with value: 0.5322751998901367 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00013058082644477008, 'epsilon': 4.4032898516212796e-05, 'beta_1': 0.9705027900866336, 'beta_2': 0.8772752078219312}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:51,334] Trial 462 finished with value: 0.43855994939804077 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00048299488712702537, 'epsilon': 4.660928845010122e-05, 'beta_1': 0.9695034599063782, 'beta_2': 0.8757316684331914}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:52,045] Trial 464 finished with value: 0.4052363336086273 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000485798342906035, 'epsilon': 7.817469801467179e-06, 'beta_1': 0.9629672136654333, 'beta_2': 0.876576366390657}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:52,665] Trial 463 finished with value: 0.5157204866409302 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00012362441426884334, 'epsilon': 7.383856707735017e-06, 'beta_1': 0.8918000707145133, 'beta_2': 0.8769674361890043}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:52:57,443] Trial 465 finished with value: 0.427091509103775 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004772952556246381, 'epsilon': 0.0011954966839279166, 'beta_1': 0.8892087867500627, 'beta_2': 0.8715683749409139}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:53:35,212] Trial 466 finished with value: 43617348.0 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.07615324914608668, 'epsilon': 7.015809775453481e-06, 'beta_1': 0.8939282176105325, 'beta_2': 0.8730001790495376}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:53:36,553] Trial 467 finished with value: 0.4363841116428375 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00043915518184492637, 'epsilon': 3.0359801291843945e-05, 'beta_1': 0.9519639750935057, 'beta_2': 0.8736446750485776}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:53:37,099] Trial 468 finished with value: 0.5172098278999329 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.03969810714955008, 'rho': 0.9109843920335677, 'momentum': 0.00017378369896760878, 'epsilon': 2.8762868731923143e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:53:38,055] Trial 469 finished with value: 0.46197956800460815 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002647296216906922, 'rho': 0.9121397348806621, 'momentum': 0.000549657183223967, 'epsilon': 2.8848130810007224e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:54:18,319] Trial 470 finished with value: 0.4531938135623932 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006153145022073489, 'rho': 0.861112985886749, 'momentum': 0.0005928592182780821, 'epsilon': 7.206224370820624e-08}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:54:19,211] Trial 471 finished with value: 0.4546564519405365 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006251453265357799, 'rho': 0.9143802680306571, 'momentum': 0.0005505519045090641, 'epsilon': 1.308809866918881e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:54:19,669] Trial 472 finished with value: 0.43228134512901306 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0007964504947731571, 'rho': 0.9385765191335322, 'momentum': 0.0005508806779985464, 'epsilon': 0.0003328303036278896}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:54:20,402] Trial 473 finished with value: 0.4213623106479645 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006404943492973721, 'rho': 0.9075693100711487, 'momentum': 0.0002646397289986362, 'epsilon': 0.0003711402118521256}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:54:59,533] Trial 474 finished with value: 0.5077561736106873 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003315769889875181, 'rho': 0.8672451167973684, 'momentum': 0.0008183033857330394, 'epsilon': 1.8784716675098734e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:01,808] Trial 476 finished with value: 0.435804545879364 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004155525381691736, 'rho': 0.9476745516074688, 'momentum': 0.00041377271770663536, 'epsilon': 1.7110365530911705e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:02,002] Trial 477 finished with value: 0.4166162610054016 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004156909951102242, 'epsilon': 0.0005763469741545735, 'beta_1': 0.9632301359325112, 'beta_2': 0.8564624458537956}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:02,034] Trial 475 finished with value: 0.43410128355026245 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000410760139866934, 'rho': 0.9077255043353809, 'momentum': 0.0002605714620837922, 'epsilon': 1.988606858199533e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:40,264] Trial 478 finished with value: 0.4250587821006775 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00043098806670512036, 'epsilon': 1.0547031570968729e-05, 'beta_1': 0.8982604522872641, 'beta_2': 0.8815956802545329}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:43,862] Trial 480 finished with value: 0.4359211027622223 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00023581275286596446, 'epsilon': 0.00012163310888263139, 'beta_1': 0.9018579602730662, 'beta_2': 0.8619338886195023}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:45,181] Trial 481 finished with value: 0.4285319745540619 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002778886030075542, 'epsilon': 3.582482118533173e-05, 'beta_1': 0.9660099034042602, 'beta_2': 0.8609437722936315}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:55:45,304] Trial 479 finished with value: 0.46983852982521057 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002590303124850884, 'epsilon': 9.404596599265175e-06, 'beta_1': 0.9626564185594338, 'beta_2': 0.8623326463801853}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:56:18,961] Trial 482 finished with value: 0.4460605978965759 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026272657846240327, 'epsilon': 3.7422556906976076e-05, 'beta_1': 0.9656880188255773, 'beta_2': 0.861014195653099}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:56:26,667] Trial 484 finished with value: 0.434934139251709 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005336172273626185, 'rho': 0.9176082756332763, 'momentum': 0.00018781311892680436, 'epsilon': 8.142720113160789e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:56:27,813] Trial 485 finished with value: 0.44724756479263306 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005328573988334461, 'rho': 0.8835783203839012, 'momentum': 0.0027047885797215443, 'epsilon': 5.599764828814107e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:56:27,831] Trial 483 finished with value: 0.4459679126739502 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003008361875985037, 'rho': 0.9576706727201452, 'momentum': 0.0021792102775965248, 'epsilon': 6.503607662632928e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:56:51,407] Trial 486 finished with value: 0.44117122888565063 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005122344606814522, 'rho': 0.9875497004372006, 'momentum': 0.002599021900612646, 'epsilon': 6.723470878017553e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:09,435] Trial 487 finished with value: 0.42100396752357483 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00034637490399587494, 'rho': 0.8832505807817045, 'momentum': 0.0022311851840862967, 'epsilon': 6.435025095301638e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:10,418] Trial 488 finished with value: 0.4194033741950989 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00036891238786822435, 'rho': 0.8804841191242024, 'momentum': 0.0014528033755158178, 'epsilon': 6.11038426531971e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:11,071] Trial 489 finished with value: 0.469970166683197 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003640268674670531, 'rho': 0.8788846111780642, 'momentum': 0.0003359011855884281, 'epsilon': 5.262168877897845e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:22,331] Trial 490 finished with value: 0.5220118761062622 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00020667386334208586, 'rho': 0.9535359714418534, 'momentum': 0.0012541206790763308, 'epsilon': 5.019339539991233e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:49,428] Trial 491 finished with value: 0.4042208194732666 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003528739464206334, 'epsilon': 5.756389343210676e-06, 'beta_1': 0.9602544483406119, 'beta_2': 0.8672790610144511}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:51,603] Trial 492 finished with value: 0.43198275566101074 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003245412496159989, 'epsilon': 5.09118871502182e-06, 'beta_1': 0.9582697293644342, 'beta_2': 0.8797614470265828}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:52,440] Trial 493 finished with value: 0.4217780530452728 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007506933314280579, 'epsilon': 1.4159441075254553e-06, 'beta_1': 0.9729534148423742, 'beta_2': 0.8671094245376114}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:57:56,577] Trial 494 finished with value: 0.42205435037612915 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00045751630151358785, 'epsilon': 2.6734983142436337e-06, 'beta_1': 0.9602631512053268, 'beta_2': 0.8668049583164689}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:58:28,983] Trial 495 finished with value: 0.42397889494895935 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046187328869018836, 'epsilon': 1.246230719609335e-05, 'beta_1': 0.959454752291645, 'beta_2': 0.8669665243850633}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:58:33,005] Trial 496 finished with value: 0.4457390308380127 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008121199927295358, 'epsilon': 9.085729998700364e-06, 'beta_1': 0.9616259371787841, 'beta_2': 0.8569654016755067}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:58:33,237] Trial 497 finished with value: 0.4409213960170746 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00047238921985806115, 'epsilon': 1.2763195706711744e-05, 'beta_1': 0.9601611489275536, 'beta_2': 0.8575239815135955}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:58:34,963] Trial 498 finished with value: 0.43555933237075806 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005682091921226161, 'epsilon': 1.3055470495513041e-05, 'beta_1': 0.9507476149821252, 'beta_2': 0.8566619653253552}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:06,167] Trial 499 finished with value: 0.4221605658531189 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006192624143547958, 'epsilon': 9.049642441761759e-06, 'beta_1': 0.9510097120718828, 'beta_2': 0.8576182632537522}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:13,821] Trial 501 finished with value: 0.418079137802124 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005807915012615691, 'epsilon': 2.034551662646567e-05, 'beta_1': 0.9517954395765098, 'beta_2': 0.8530923771349908}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:13,845] Trial 502 finished with value: 0.4348512589931488 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00038544876469821506, 'epsilon': 3.548854929440399e-05, 'beta_1': 0.9679062816692743, 'beta_2': 0.8861110952162344}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:13,849] Trial 500 finished with value: 0.43766897916793823 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005857001034640003, 'epsilon': 3.566376461664903e-05, 'beta_1': 0.952296983921927, 'beta_2': 0.8869405512563608}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:37,629] Trial 503 finished with value: 0.4602143168449402 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000371290079185236, 'epsilon': 3.648042883634197e-05, 'beta_1': 0.8806127431603221, 'beta_2': 0.8537325500927891}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:56,798] Trial 505 finished with value: 0.42382389307022095 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00034343530108367503, 'epsilon': 3.490733443939856e-06, 'beta_1': 0.9707793125630616, 'beta_2': 0.8708132076963141}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:56,822] Trial 506 finished with value: 0.42053109407424927 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003351797735174613, 'epsilon': 0.00013918627787340207, 'beta_1': 0.9710009106722517, 'beta_2': 0.8703336612702885}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 20:59:56,857] Trial 504 finished with value: 0.41767072677612305 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003567242528042026, 'epsilon': 3.6864605021622126e-06, 'beta_1': 0.9749709195990912, 'beta_2': 0.870609911282717}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:00:09,530] Trial 507 finished with value: 0.426107257604599 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032462615540689737, 'epsilon': 0.00014680727129419598, 'beta_1': 0.9710919053880854, 'beta_2': 0.8502609056596175}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:00:38,757] Trial 509 finished with value: 0.4385085105895996 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042375693592684315, 'epsilon': 2.459240092281576e-05, 'beta_1': 0.9468805226744037, 'beta_2': 0.8642895532528753}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:00:40,669] Trial 510 finished with value: 0.4182441234588623 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004465596098792901, 'epsilon': 2.0171035879516132e-05, 'beta_1': 0.9418687021086053, 'beta_2': 0.8640837855888154}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:00:40,708] Trial 508 finished with value: 0.4499727189540863 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00043228244028566115, 'epsilon': 2.4717089649765778e-05, 'beta_1': 0.9820169567111167, 'beta_2': 0.8637264985500278}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:00:45,455] Trial 511 finished with value: 0.4223921298980713 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00044732735855284927, 'epsilon': 7.501663781652126e-06, 'beta_1': 0.9551070212873525, 'beta_2': 0.8656778423674455}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:01:22,643] Trial 512 finished with value: 0.44065794348716736 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005006951004801469, 'epsilon': 1.9272980490717942e-05, 'beta_1': 0.9824441843400188, 'beta_2': 0.8500504393268526}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:01:25,975] Trial 513 finished with value: 0.4156857430934906 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026396575577451183, 'epsilon': 6.808818953524579e-06, 'beta_1': 0.9667606342558174, 'beta_2': 0.8589247638697644}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:01:26,623] Trial 514 finished with value: 0.6015007495880127 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00015865455057604086, 'rho': 0.8621560863767787, 'momentum': 0.09860683663794584, 'epsilon': 0.0005407818600424895}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:01:28,149] Trial 515 finished with value: 0.44520989060401917 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00022061747574645836, 'rho': 0.9756634261445489, 'momentum': 0.0818454788009766, 'epsilon': 0.00027704113760975406}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:06,241] Trial 516 finished with value: 0.43400412797927856 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00027919075318734606, 'rho': 0.8940759473956386, 'momentum': 6.964070618256684e-05, 'epsilon': 0.00026163278270608894}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:09,046] Trial 517 finished with value: 0.4801790714263916 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00020730512605641094, 'rho': 0.8614376530033915, 'momentum': 0.09921900451359064, 'epsilon': 5.025883470323733e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:10,509] Trial 519 finished with value: 0.4754274785518646 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002868909149012473, 'rho': 0.8928107491536706, 'momentum': 0.0010633411070077598, 'epsilon': 5.147096700729557e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:10,537] Trial 518 finished with value: 0.42148298025131226 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00022375921204354128, 'rho': 0.9899656224428423, 'momentum': 0.00041486750723020247, 'epsilon': 4.9929959925661585e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:45,677] Trial 520 finished with value: 0.43741756677627563 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006907679642510179, 'rho': 0.8662071707244209, 'momentum': 0.00041359629359829446, 'epsilon': 8.824522099239513e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:52,000] Trial 521 finished with value: 0.4185361862182617 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003083845985564185, 'rho': 0.8725740295402005, 'momentum': 0.0010542021569571165, 'epsilon': 1.5411112115832862e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:53,117] Trial 522 finished with value: 0.4194755554199219 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005465071159939661, 'rho': 0.9844397601311293, 'momentum': 0.0008762666815034706, 'epsilon': 0.004055963980175245}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:02:53,390] Trial 523 finished with value: 0.453541100025177 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007300277526474103, 'epsilon': 9.366134961109527e-05, 'beta_1': 0.9637148653125319, 'beta_2': 0.8789687540788417}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:03:17,596] Trial 524 finished with value: 0.4256196618080139 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005268753698024308, 'epsilon': 1.4654522215298911e-05, 'beta_1': 0.9751715268893087, 'beta_2': 0.8744577730927768}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:03:33,553] Trial 525 finished with value: 0.421541690826416 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005506252481651584, 'epsilon': 0.004107396739302709, 'beta_1': 0.9754609651558349, 'beta_2': 0.8792436357865822}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:03:34,814] Trial 526 finished with value: 0.4277488589286804 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003825924343649374, 'epsilon': 2.6688367750565144e-05, 'beta_1': 0.9648674861214265, 'beta_2': 0.8746644874571861}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:03:35,169] Trial 527 finished with value: 0.4258976876735687 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000950405354556427, 'epsilon': 2.834919317370501e-05, 'beta_1': 0.9648447009404502, 'beta_2': 0.8914085639310889}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:03:47,186] Trial 528 finished with value: 0.42322173714637756 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00040106722237716704, 'epsilon': 2.8758266919590683e-05, 'beta_1': 0.9569378750669926, 'beta_2': 0.890721030683307}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:14,808] Trial 529 finished with value: 0.4182116389274597 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00037559475677291076, 'epsilon': 2.9710194609405036e-05, 'beta_1': 0.9864041265163391, 'beta_2': 0.8899503249574938}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:16,222] Trial 530 finished with value: 0.4192669987678528 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00039702688487298985, 'rho': 0.9720573984208927, 'momentum': 0.06988958248992076, 'epsilon': 1.2899037302248798e-07}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:16,271] Trial 531 finished with value: 0.5079513788223267 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000378001664202454, 'rho': 0.8686729952175141, 'momentum': 0.06706227110131331, 'epsilon': 2.835157416705168e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:21,209] Trial 532 finished with value: 0.4156418442726135 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003120676459086562, 'rho': 0.9442454517210062, 'momentum': 0.00023924592875587252, 'epsilon': 2.684325225872392e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:55,195] Trial 533 finished with value: 0.43363314867019653 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003132652788011491, 'rho': 0.855521226012871, 'momentum': 0.00016195735640077297, 'epsilon': 3.873472701329135e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:57,624] Trial 534 finished with value: 0.42722901701927185 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003063105488062435, 'rho': 0.8687399834460409, 'momentum': 0.00013232603145785208, 'epsilon': 3.8847538023319916e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:04:58,725] Trial 535 finished with value: 5.5845794677734375 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00031472458651339666, 'rho': 0.9274870267066263}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:05:00,326] Trial 536 finished with value: 0.40288567543029785 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005081640777865783, 'rho': 0.8546355305006188, 'momentum': 4.54503000311247e-05, 'epsilon': 1.7739802893665101e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:05:35,670] Trial 537 finished with value: 5.609253883361816 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00045465374767486555, 'rho': 0.9280823854507289}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:05:38,369] Trial 538 finished with value: 0.42351269721984863 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004582345791481698, 'epsilon': 9.8704519009491e-06, 'beta_1': 0.9105504310976844, 'beta_2': 0.8529739198954882}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:05:39,204] Trial 539 finished with value: 0.4115280210971832 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00047959165119832754, 'epsilon': 7.792628391454624e-07, 'beta_1': 0.9135576923952784, 'beta_2': 0.8544511685799302}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:05:39,461] Trial 540 finished with value: 0.4466204047203064 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006704970896206814, 'epsilon': 3.738821919870398e-05, 'beta_1': 0.983879066285477, 'beta_2': 0.8546389827263898}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:06:16,954] Trial 541 finished with value: 0.4288574755191803 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006972690203416811, 'epsilon': 1.7873530739822429e-06, 'beta_1': 0.9843501721440461, 'beta_2': 0.8529247355717363}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:06:19,603] Trial 542 finished with value: 0.4318903088569641 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000682567860782025, 'epsilon': 1.6501476996643781e-06, 'beta_1': 0.9678131200100294, 'beta_2': 0.8826340016662595}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:06:20,223] Trial 543 finished with value: 0.41662949323654175 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007432970823669176, 'epsilon': 1.790840914326609e-06, 'beta_1': 0.9677049717929381, 'beta_2': 0.8551270518996792}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:06:20,247] Trial 544 finished with value: 0.433828741312027 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006003797436734479, 'epsilon': 7.556544529661122e-07, 'beta_1': 0.8684341831789809, 'beta_2': 0.8596432974825896}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:06:57,235] Trial 545 finished with value: 0.47032323479652405 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000614856432741529, 'epsilon': 7.275099791660178e-07, 'beta_1': 0.8688378006660524, 'beta_2': 0.8601430316867926}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:01,540] Trial 546 finished with value: 0.4176555275917053 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005372989766997289, 'epsilon': 5.93410151551918e-07, 'beta_1': 0.9061517396167663, 'beta_2': 0.8500712840455392}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:01,573] Trial 548 finished with value: 0.4253268837928772 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005142752161843908, 'epsilon': 8.905703206869272e-07, 'beta_1': 0.9046113544023255, 'beta_2': 0.8686016576705201}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:01,642] Trial 547 finished with value: 0.4447399973869324 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000502814805310223, 'epsilon': 9.872839036828515e-07, 'beta_1': 0.8747655281909952, 'beta_2': 0.8503103668573477}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:38,268] Trial 549 finished with value: 0.4320693910121918 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005035459370621422, 'epsilon': 1.2096950292603844e-06, 'beta_1': 0.9153840921585662, 'beta_2': 0.8500409618375212}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:43,853] Trial 552 finished with value: 0.46510785818099976 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00025329463688974487, 'epsilon': 2.0633502423898407e-06, 'beta_1': 0.9546416641994081, 'beta_2': 0.8611268499224299}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:43,961] Trial 550 finished with value: 0.4298473000526428 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005089382425752561, 'epsilon': 2.139008696649394e-07, 'beta_1': 0.9614724980348014, 'beta_2': 0.8623180437052207}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:07:44,013] Trial 551 finished with value: 0.44419723749160767 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00024811518005485056, 'epsilon': 9.286043149697372e-07, 'beta_1': 0.9582111775432355, 'beta_2': 0.858720712444261}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:08:12,581] Trial 553 finished with value: 0.4435047209262848 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009203138809259256, 'rho': 0.8500020463325494, 'momentum': 3.7432358826999286e-05, 'epsilon': 1.3695139146085507e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:08:27,448] Trial 554 finished with value: 0.4359171390533447 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003952505084582648, 'rho': 0.8543977240391362, 'momentum': 0.00032314559880180165, 'epsilon': 1.4072662556157728e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:08:27,590] Trial 555 finished with value: 0.4302182197570801 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0008921887894403382, 'rho': 0.8536829508819668, 'momentum': 0.000648201846871886, 'epsilon': 1.4153306898064368e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:08:27,667] Trial 556 finished with value: 0.45527759194374084 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0008679148932151114, 'rho': 0.8976813537886975, 'momentum': 4.5973617257028675e-05, 'epsilon': 1.2558153382897266e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:08:43,630] Trial 557 finished with value: 0.41176533699035645 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00037828561326218687, 'rho': 0.8569249227579369, 'momentum': 6.585390004661946e-05, 'epsilon': 6.908151169837047e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:10,946] Trial 559 finished with value: 0.4377896189689636 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00035974042429331624, 'rho': 0.8764238234649957, 'momentum': 0.0006977633569539717, 'epsilon': 0.009245784502454392}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:11,075] Trial 560 finished with value: 0.4975166916847229 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003473840770739682, 'rho': 0.8601399039628065, 'momentum': 2.16647334200169e-05, 'epsilon': 6.834620753106568e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:11,168] Trial 558 finished with value: 0.42039719223976135 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00036791741806804464, 'rho': 0.8599836605640689, 'momentum': 0.000488728250167534, 'epsilon': 6.740640477875408e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:19,553] Trial 561 finished with value: 0.4307931065559387 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003559769587372274, 'epsilon': 4.630502910795527e-05, 'beta_1': 0.9775108932977485, 'beta_2': 0.8557890599882055}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:55,909] Trial 562 finished with value: 0.5216324925422668 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00019229183388911987, 'epsilon': 2.772647050241881e-08, 'beta_1': 0.9896624251898202, 'beta_2': 0.8565790599984843}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:55,939] Trial 563 finished with value: 0.5214737057685852 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001824509946730033, 'epsilon': 0.000866789666134492, 'beta_1': 0.9730520283920506, 'beta_2': 0.9748827506731669}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:55,943] Trial 564 finished with value: 0.4569271206855774 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004538653177955547, 'epsilon': 4.348447632410779e-05, 'beta_1': 0.9454874284070113, 'beta_2': 0.8558449853579877}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:09:58,050] Trial 565 finished with value: 4.421084880828857 and parameters: {'optimizer': 'Adam', 'learning_rate': 3.563317508534132e-05, 'epsilon': 2.8310477801779888e-08, 'beta_1': 0.9723722093789287, 'beta_2': 0.8685601162595914}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:10:36,713] Trial 568 finished with value: 0.41358327865600586 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00044552163331658157, 'rho': 0.8650771692008881, 'momentum': 0.0008273113218205385, 'epsilon': 2.3544342189190232e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:10:38,070] Trial 569 finished with value: 0.42934417724609375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00025830204645219183, 'rho': 0.980470010637668, 'momentum': 0.0017583658605317737, 'epsilon': 2.3341464674878646e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:10:38,185] Trial 566 finished with value: 0.40777674317359924 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004508491777239988, 'epsilon': 2.317716084511032e-06, 'beta_1': 0.9625838993943611, 'beta_2': 0.8683701073911542}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:10:38,230] Trial 567 finished with value: 0.4187648892402649 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000268484610560277, 'rho': 0.8642293082520836, 'momentum': 2.491624891891473e-05, 'epsilon': 2.5862855400021567e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:11:17,301] Trial 570 finished with value: 0.4940313398838043 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00027617372024840654, 'rho': 0.979742516565487, 'momentum': 1.518657531488967e-05, 'epsilon': 2.041848796064605e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:11:19,549] Trial 573 finished with value: 0.4217144548892975 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005949246537862604, 'epsilon': 3.590263059431128e-05, 'beta_1': 0.962628430531134, 'beta_2': 0.8731534555134385}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:11:19,644] Trial 571 finished with value: 0.4398156702518463 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005812713738590779, 'epsilon': 2.3049631303927343e-05, 'beta_1': 0.8970340442739295, 'beta_2': 0.8723236472317872}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:11:19,663] Trial 572 finished with value: 0.42494773864746094 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028236552734277093, 'rho': 0.9236785387426344, 'momentum': 1.0819411643366565e-05, 'epsilon': 1.7586666361780788e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:11:58,178] Trial 574 finished with value: 0.43468326330184937 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000555140573131636, 'epsilon': 0.0017190199462480973, 'beta_1': 0.9642892043434614, 'beta_2': 0.8663511823310264}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:00,018] Trial 575 finished with value: 0.4280228018760681 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00043017256495724986, 'rho': 0.8705250323271249, 'momentum': 3.053083275203286e-05, 'epsilon': 0.00045717230620360424}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:00,419] Trial 576 finished with value: 0.44124865531921387 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00042793121175035856, 'rho': 0.871314062098702, 'momentum': 0.012496863389867906, 'epsilon': 3.042143905389507e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:00,637] Trial 577 finished with value: 0.42367133498191833 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000423422436703273, 'rho': 0.8707312374248829, 'momentum': 3.4643436891537786e-05, 'epsilon': 3.2491386901435563e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:38,771] Trial 578 finished with value: 0.4431585371494293 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00042949718173422845, 'rho': 0.871909127056862, 'momentum': 5.068677406883776e-05, 'epsilon': 3.5337051876122486e-06}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:41,343] Trial 580 finished with value: 0.4250589907169342 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00031494592440034274, 'epsilon': 5.698155667790772e-06, 'beta_1': 0.9692456063622559, 'beta_2': 0.8775772338726965}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:41,389] Trial 581 finished with value: 0.4087948799133301 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003154927768634133, 'epsilon': 0.00011225062561970816, 'beta_1': 0.9683845150886187, 'beta_2': 0.87678944318605}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:12:41,445] Trial 579 finished with value: 0.4201219975948334 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004237647892868596, 'rho': 0.8731603002871766, 'momentum': 7.71859080952002e-05, 'epsilon': 3.716668176931509e-05}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:13:16,894] Trial 582 finished with value: 0.4178386330604553 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003150645525542319, 'epsilon': 4.388384690990877e-05, 'beta_1': 0.9686852671623899, 'beta_2': 0.8628763960444893}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:13:21,621] Trial 584 finished with value: 0.465639591217041 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00021069279522019992, 'epsilon': 0.00019090199735418282, 'beta_1': 0.9582742891345533, 'beta_2': 0.8837625817379134}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:13:21,622] Trial 583 finished with value: 0.46317410469055176 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000219691812123445, 'epsilon': 5.2330486538325255e-05, 'beta_1': 0.9588805538633697, 'beta_2': 0.8853648217481832}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:13:21,676] Trial 585 finished with value: 0.43811383843421936 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00022496318859713501, 'epsilon': 0.00021678581773359844, 'beta_1': 0.9589293649575287, 'beta_2': 0.8816291345097436}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:13:49,587] Trial 586 finished with value: 0.45250287652015686 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002595361062264275, 'epsilon': 0.00010867025427397982, 'beta_1': 0.9576267667271713, 'beta_2': 0.883776005468558}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:04,305] Trial 587 finished with value: 0.43695348501205444 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003356965287026852, 'epsilon': 8.414600106044018e-05, 'beta_1': 0.9620530166168935, 'beta_2': 0.868970739815677}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:04,340] Trial 589 finished with value: 0.422433078289032 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00031786189430667083, 'epsilon': 0.00012166710177554346, 'beta_1': 0.9548143149809607, 'beta_2': 0.8756813646256402}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:04,379] Trial 588 finished with value: 0.43665704131126404 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00024248311502647008, 'epsilon': 0.0001303652834922017, 'beta_1': 0.9661594091763321, 'beta_2': 0.8694469915140325}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:20,196] Trial 590 finished with value: 0.43653711676597595 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00033299963601114403, 'epsilon': 0.0001288575247518993, 'beta_1': 0.9647222148552247, 'beta_2': 0.8719100219945205}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:46,940] Trial 592 finished with value: 0.5142552852630615 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00016189382585247072, 'epsilon': 0.0006463860583665241, 'beta_1': 0.9739510597161004, 'beta_2': 0.8656818461462651}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:47,109] Trial 591 finished with value: 0.40701502561569214 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00031348064965353506, 'epsilon': 0.00011934810316297615, 'beta_1': 0.9655108632664274, 'beta_2': 0.8650360741807976}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:47,810] Trial 593 finished with value: 0.43693462014198303 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000360990885922431, 'epsilon': 0.00040304269466653967, 'beta_1': 0.9717838878025845, 'beta_2': 0.8956268992265656}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:14:55,240] Trial 594 finished with value: 0.4384438991546631 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005051712166215217, 'epsilon': 8.009366089353015e-05, 'beta_1': 0.9485687637415381, 'beta_2': 0.8961164442713834}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:15:29,870] Trial 597 finished with value: 0.4168674349784851 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00027669885395820165, 'epsilon': 8.347099380027336e-05, 'beta_1': 0.9778262478016752, 'beta_2': 0.8762854830869192}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:15:30,988] Trial 595 finished with value: 0.44044479727745056 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005027318630157354, 'epsilon': 8.649615419765218e-05, 'beta_1': 0.9778552685796149, 'beta_2': 0.8971979093784223}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:15:31,002] Trial 596 finished with value: 0.4157314896583557 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003021176503327234, 'epsilon': 9.227363376344869e-05, 'beta_1': 0.98094378400486, 'beta_2': 0.8967189520862551}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:15:31,611] Trial 598 finished with value: 0.4422950744628906 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000284428435340234, 'epsilon': 7.509320976147377e-05, 'beta_1': 0.977249112488647, 'beta_2': 0.8766211374975865}. Best is trial 345 with value: 0.38822034001350403. [I 2021-03-10 21:15:48,652] Trial 599 finished with value: 0.4885005056858063 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006617661422010057, 'epsilon': 6.0247893708130986e-05, 'beta_1': 0.9705655217688528, 'beta_2': 0.86358917780075}. Best is trial 345 with value: 0.38822034001350403.
fine_best_params,opt_conf = study_fine.best_params,study_fine.best_params
fine_best_params, study_fine.best_value
({'optimizer': 'Adam',
'learning_rate': 0.00035718724976417535,
'epsilon': 3.5043808140212366e-05,
'beta_1': 0.9834335585011938,
'beta_2': 0.8583875028859446},
0.38822034001350403)
hp_score_board = hp_score_board.append({'Phase':'Fine Tuning - Optimizer',
'Best Value':study_fine.best_value,
'Hyperparameters':f'{study_fine.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.398870 | {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256} |
| 1 | Coarse Tuning | 0.399832 | {'drop_l1': 0.15872829858660079, 'drop_l2': 0.124404021283663, 'drop_l3': 0.1191444759931971, 'drop_l4': 0.6775070097550181, 'drop_l5': 0.32337913815894676, 'drop_l6': 0.13903817315530098, 'drop_l7': 0.39426663755264946, 'activation': 'relu', 'kernel_init': 'uniform'} |
| 2 | Fine Tuning - Optimizer | 0.388220 | {'optimizer': 'Adam', 'learning_rate': 0.00035718724976417535, 'epsilon': 3.5043808140212366e-05, 'beta_1': 0.9834335585011938, 'beta_2': 0.8583875028859446} |
0.01 with Adam optimizer and learning rate of 0.0003.## building the optimizer from the fine tuning.
sel_opti = fine_best_params['optimizer']
fine_best_params.pop('optimizer', None)
fine_best_params
optimizer = getattr(tf.optimizers, sel_opti)(**fine_best_params)
optimizer
<tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7f3087be0430>
hp_best_params['opts'] = optimizer
hp_best_params
{'num_layers': 7,
'arch': 'D A B',
'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256],
'activation': 'relu',
'kernel_init': 'uniform',
'drop_rate_per_layer': [0.15872829858660079,
0.124404021283663,
0.1191444759931971,
0.6775070097550181,
0.32337913815894676,
0.13903817315530098,
0.39426663755264946],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7f3087be0430>}
study_fine.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_beta_1 | params_beta_2 | params_epsilon | params_learning_rate | params_momentum | params_nesterov | params_optimizer | params_rho | state | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.669406 | 2021-03-10 19:34:04.674617 | 2021-03-10 19:34:46.235076 | 0 days 00:00:41.560459 | 0.985645 | 0.975730 | 6.102474e-07 | 0.087726 | NaN | NaN | Adam | NaN | COMPLETE |
| 1 | 1 | 0.431048 | 2021-03-10 19:34:04.675775 | 2021-03-10 19:34:46.408579 | 0 days 00:00:41.732804 | 0.851893 | 0.967346 | 1.496667e-04 | 0.000518 | NaN | NaN | Adam | NaN | COMPLETE |
| 2 | 2 | 0.637111 | 2021-03-10 19:34:04.677828 | 2021-03-10 19:34:46.373833 | 0 days 00:00:41.696005 | NaN | NaN | NaN | 0.018972 | NaN | NaN | Adadelta | 0.91044 | COMPLETE |
| 3 | 3 | 0.496351 | 2021-03-10 19:34:04.680569 | 2021-03-10 19:34:44.048072 | 0 days 00:00:39.367503 | 0.911663 | 0.864106 | 3.773701e-07 | 0.000150 | NaN | NaN | Adam | NaN | COMPLETE |
| 4 | 4 | 0.446914 | 2021-03-10 19:34:44.052128 | 2021-03-10 19:35:24.065508 | 0 days 00:00:40.013380 | 0.921803 | 0.896630 | 1.903494e-07 | 0.000600 | NaN | NaN | Adam | NaN | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 595 | 595 | 0.440445 | 2021-03-10 21:14:46.945368 | 2021-03-10 21:15:30.987853 | 0 days 00:00:44.042485 | 0.977855 | 0.897198 | 8.649615e-05 | 0.000503 | NaN | NaN | Adam | NaN | COMPLETE |
| 596 | 596 | 0.415731 | 2021-03-10 21:14:47.121588 | 2021-03-10 21:15:31.002454 | 0 days 00:00:43.880866 | 0.980944 | 0.896719 | 9.227363e-05 | 0.000302 | NaN | NaN | Adam | NaN | COMPLETE |
| 597 | 597 | 0.416867 | 2021-03-10 21:14:47.815508 | 2021-03-10 21:15:29.870096 | 0 days 00:00:42.054588 | 0.977826 | 0.876285 | 8.347099e-05 | 0.000277 | NaN | NaN | Adam | NaN | COMPLETE |
| 598 | 598 | 0.442295 | 2021-03-10 21:14:55.243054 | 2021-03-10 21:15:31.611291 | 0 days 00:00:36.368237 | 0.977249 | 0.876621 | 7.509321e-05 | 0.000284 | NaN | NaN | Adam | NaN | COMPLETE |
| 599 | 599 | 0.488501 | 2021-03-10 21:15:29.879919 | 2021-03-10 21:15:48.651686 | 0 days 00:00:18.771767 | 0.970566 | 0.863589 | 6.024789e-05 | 0.000662 | NaN | NaN | Adam | NaN | COMPLETE |
600 rows × 14 columns
optuna.visualization.plot_parallel_coordinate(study_fine)
optuna.visualization.plot_optimization_history(study_fine)
optuna.visualization.plot_slice(study_fine)
hp_best_params
{'num_layers': 7,
'arch': 'D A B',
'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256],
'activation': 'relu',
'kernel_init': 'uniform',
'drop_rate_per_layer': [0.15872829858660079,
0.124404021283663,
0.1191444759931971,
0.6775070097550181,
0.32337913815894676,
0.13903817315530098,
0.39426663755264946],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7f3087be0430>}
def objective_final(trial):
global hp_best_params,loss,metric,X_train,X_test,y_train,y_test
# momentum = trial.suggest_uniform('momentum',0.9,0.99)
# epsilon = trial.suggest_loguniform('epsilon',0.0005,0.05)
# beta_init_std = trial.suggest_categorical('beta_init_std',[1.0,0.5,0.05,0.005,0.005])
# gamma_init = trial.suggest_categorical('gamma_init',[0.8,0.9,1.0])
# center = trial.suggest_categorical('center',[True,False])
# scale = trial.suggest_categorical('scale',[True,False])
batch_size = trial.suggest_int('batch_size',1,100)
n_epochs = trial.suggest_categorical('n_epochs',[50,100,200,500,1000,2000])
kwargs = {
'batch_size' : batch_size,
'n_epochs' : n_epochs
}
kwargs = {**hp_best_params, **kwargs}
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1)
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,['mean_squared_error'],['mean_squared_error'],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[0]
print(X_train.shape,X_test.shape,y_train.shape,y_test.shape)
study_final = optuna.create_study(direction='minimize',sampler=optuna.samplers.TPESampler())
study_final.optimize(objective_final,n_trials=500,n_jobs=-13)
[I 2021-03-10 21:15:49,571] A new study created in memory with name: no-name-9919e019-bc03-497a-ae2c-914aef065841 [I 2021-03-10 21:16:07,232] Trial 0 finished with value: 0.5533801913261414 and parameters: {'batch_size': 97, 'n_epochs': 200}. Best is trial 0 with value: 0.5533801913261414. [I 2021-03-10 21:16:07,250] Trial 1 finished with value: 0.5033710598945618 and parameters: {'batch_size': 97, 'n_epochs': 100}. Best is trial 1 with value: 0.5033710598945618. [I 2021-03-10 21:16:07,302] Trial 2 finished with value: 0.5196731090545654 and parameters: {'batch_size': 75, 'n_epochs': 100}. Best is trial 1 with value: 0.5033710598945618. [I 2021-03-10 21:16:07,318] Trial 3 finished with value: 0.5424050092697144 and parameters: {'batch_size': 69, 'n_epochs': 2000}. Best is trial 1 with value: 0.5033710598945618. [I 2021-03-10 21:16:24,925] Trial 4 finished with value: 0.49471673369407654 and parameters: {'batch_size': 77, 'n_epochs': 50}. Best is trial 4 with value: 0.49471673369407654. [I 2021-03-10 21:16:25,230] Trial 5 finished with value: 0.48475807905197144 and parameters: {'batch_size': 41, 'n_epochs': 2000}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:16:25,303] Trial 6 finished with value: 0.5395416617393494 and parameters: {'batch_size': 14, 'n_epochs': 50}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:16:25,334] Trial 7 finished with value: 0.5218173265457153 and parameters: {'batch_size': 3, 'n_epochs': 200}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:16:42,630] Trial 8 finished with value: 0.5291265249252319 and parameters: {'batch_size': 47, 'n_epochs': 500}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:16:43,028] Trial 11 finished with value: 0.5290259122848511 and parameters: {'batch_size': 22, 'n_epochs': 200}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:16:43,063] Trial 10 finished with value: 0.5155233144760132 and parameters: {'batch_size': 63, 'n_epochs': 500}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:16:43,073] Trial 9 finished with value: 0.505242645740509 and parameters: {'batch_size': 32, 'n_epochs': 200}. Best is trial 5 with value: 0.48475807905197144. [I 2021-03-10 21:17:00,115] Trial 12 finished with value: 0.4742027819156647 and parameters: {'batch_size': 72, 'n_epochs': 200}. Best is trial 12 with value: 0.4742027819156647. [I 2021-03-10 21:17:00,998] Trial 13 finished with value: 0.47304636240005493 and parameters: {'batch_size': 40, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:01,453] Trial 15 finished with value: 0.5052279829978943 and parameters: {'batch_size': 46, 'n_epochs': 1000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:01,757] Trial 14 finished with value: 0.5599774718284607 and parameters: {'batch_size': 48, 'n_epochs': 1000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:17,958] Trial 16 finished with value: 0.5081806182861328 and parameters: {'batch_size': 48, 'n_epochs': 1000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:18,036] Trial 17 finished with value: 0.513993501663208 and parameters: {'batch_size': 58, 'n_epochs': 1000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:18,649] Trial 18 finished with value: 0.48730865120887756 and parameters: {'batch_size': 59, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:18,695] Trial 19 finished with value: 0.5089763402938843 and parameters: {'batch_size': 61, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:35,874] Trial 20 finished with value: 0.4755844473838806 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:36,131] Trial 21 finished with value: 0.5216754674911499 and parameters: {'batch_size': 81, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:36,407] Trial 23 finished with value: 0.4786377549171448 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:36,424] Trial 22 finished with value: 0.528331995010376 and parameters: {'batch_size': 85, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:51,863] Trial 25 finished with value: 0.5536627769470215 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:51,907] Trial 27 finished with value: 0.499502956867218 and parameters: {'batch_size': 91, 'n_epochs': 200}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:51,912] Trial 24 finished with value: 0.6099499464035034 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 13 with value: 0.47304636240005493. [I 2021-03-10 21:17:51,923] Trial 26 finished with value: 0.46606284379959106 and parameters: {'batch_size': 89, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:09,739] Trial 30 finished with value: 0.5157343149185181 and parameters: {'batch_size': 32, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:09,774] Trial 31 finished with value: 0.5195454359054565 and parameters: {'batch_size': 36, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:10,136] Trial 28 finished with value: 0.5510433912277222 and parameters: {'batch_size': 32, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:11,179] Trial 29 finished with value: 0.6221821308135986 and parameters: {'batch_size': 30, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:26,157] Trial 32 finished with value: 0.4977339208126068 and parameters: {'batch_size': 70, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:26,178] Trial 34 finished with value: 0.4756341576576233 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:26,183] Trial 33 finished with value: 0.488224595785141 and parameters: {'batch_size': 97, 'n_epochs': 200}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:26,666] Trial 35 finished with value: 0.5021785497665405 and parameters: {'batch_size': 100, 'n_epochs': 100}. Best is trial 26 with value: 0.46606284379959106. [I 2021-03-10 21:18:43,557] Trial 36 finished with value: 0.43997272849082947 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:18:43,627] Trial 38 finished with value: 0.5128291845321655 and parameters: {'batch_size': 93, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:18:43,635] Trial 39 finished with value: 0.5300602912902832 and parameters: {'batch_size': 73, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:18:45,558] Trial 37 finished with value: 0.5596278309822083 and parameters: {'batch_size': 72, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:18:59,435] Trial 40 finished with value: 0.5969325304031372 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:18:59,826] Trial 42 finished with value: 0.4937376081943512 and parameters: {'batch_size': 55, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:18:59,829] Trial 41 finished with value: 0.5095510482788086 and parameters: {'batch_size': 78, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:00,368] Trial 43 finished with value: 0.5017756223678589 and parameters: {'batch_size': 54, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:17,118] Trial 44 finished with value: 0.5430660843849182 and parameters: {'batch_size': 78, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:17,759] Trial 46 finished with value: 0.5583347082138062 and parameters: {'batch_size': 81, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:18,130] Trial 45 finished with value: 0.5357880592346191 and parameters: {'batch_size': 81, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:18,964] Trial 47 finished with value: 0.4793166518211365 and parameters: {'batch_size': 79, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:34,926] Trial 48 finished with value: 0.583471417427063 and parameters: {'batch_size': 65, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:35,070] Trial 50 finished with value: 0.48498669266700745 and parameters: {'batch_size': 67, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:35,079] Trial 49 finished with value: 0.5155723094940186 and parameters: {'batch_size': 65, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:35,676] Trial 51 finished with value: 0.580171525478363 and parameters: {'batch_size': 89, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:52,750] Trial 52 finished with value: 0.4508623480796814 and parameters: {'batch_size': 42, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:54,233] Trial 54 finished with value: 0.526282548904419 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:54,274] Trial 53 finished with value: 0.5188635587692261 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:19:54,346] Trial 55 finished with value: 0.5787612795829773 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:10,474] Trial 56 finished with value: 0.544662356376648 and parameters: {'batch_size': 40, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:11,498] Trial 58 finished with value: 0.4660778045654297 and parameters: {'batch_size': 42, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:11,573] Trial 57 finished with value: 0.4822506308555603 and parameters: {'batch_size': 44, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:11,634] Trial 59 finished with value: 0.5306397676467896 and parameters: {'batch_size': 44, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:29,601] Trial 60 finished with value: 0.5121575593948364 and parameters: {'batch_size': 42, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:31,055] Trial 61 finished with value: 0.47301357984542847 and parameters: {'batch_size': 25, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:31,064] Trial 62 finished with value: 0.5427552461624146 and parameters: {'batch_size': 24, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:31,088] Trial 63 finished with value: 0.4923693537712097 and parameters: {'batch_size': 24, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:46,583] Trial 64 finished with value: 0.4979555010795593 and parameters: {'batch_size': 26, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:47,367] Trial 66 finished with value: 0.5888012051582336 and parameters: {'batch_size': 20, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:47,744] Trial 67 finished with value: 0.46704334020614624 and parameters: {'batch_size': 17, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:20:48,197] Trial 65 finished with value: 0.515156626701355 and parameters: {'batch_size': 16, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:07,099] Trial 68 finished with value: 0.5222811698913574 and parameters: {'batch_size': 3, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:07,654] Trial 69 finished with value: 0.5472179055213928 and parameters: {'batch_size': 16, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:07,657] Trial 70 finished with value: 0.4874271750450134 and parameters: {'batch_size': 2, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:08,781] Trial 71 finished with value: 0.5105281472206116 and parameters: {'batch_size': 10, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:23,088] Trial 72 finished with value: 0.5028771162033081 and parameters: {'batch_size': 9, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:23,484] Trial 74 finished with value: 0.5227941870689392 and parameters: {'batch_size': 12, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:23,540] Trial 73 finished with value: 0.5454003810882568 and parameters: {'batch_size': 39, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:24,756] Trial 75 finished with value: 0.49138349294662476 and parameters: {'batch_size': 36, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:42,982] Trial 76 finished with value: 0.518235981464386 and parameters: {'batch_size': 50, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:43,567] Trial 78 finished with value: 0.5808281898498535 and parameters: {'batch_size': 36, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:43,960] Trial 77 finished with value: 0.5466302633285522 and parameters: {'batch_size': 37, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:44,473] Trial 79 finished with value: 0.558784008026123 and parameters: {'batch_size': 36, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:58,189] Trial 80 finished with value: 0.46553540229797363 and parameters: {'batch_size': 29, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:58,794] Trial 81 finished with value: 0.5080075860023499 and parameters: {'batch_size': 27, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:21:59,261] Trial 82 finished with value: 0.537264883518219 and parameters: {'batch_size': 31, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:00,031] Trial 83 finished with value: 0.5789027810096741 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:18,565] Trial 85 finished with value: 0.5873022079467773 and parameters: {'batch_size': 27, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:18,568] Trial 86 finished with value: 0.4934960901737213 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:18,570] Trial 84 finished with value: 0.5075036883354187 and parameters: {'batch_size': 29, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:19,370] Trial 87 finished with value: 0.4872015118598938 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:32,090] Trial 90 finished with value: 0.5414965748786926 and parameters: {'batch_size': 52, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:32,107] Trial 88 finished with value: 0.49623632431030273 and parameters: {'batch_size': 50, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:32,295] Trial 89 finished with value: 0.5350531935691833 and parameters: {'batch_size': 50, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:33,298] Trial 91 finished with value: 0.508921205997467 and parameters: {'batch_size': 50, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:51,696] Trial 94 finished with value: 0.48744821548461914 and parameters: {'batch_size': 89, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:51,708] Trial 93 finished with value: 0.5862656831741333 and parameters: {'batch_size': 46, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:52,100] Trial 92 finished with value: 0.5267103910446167 and parameters: {'batch_size': 44, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:22:52,500] Trial 95 finished with value: 0.5452749729156494 and parameters: {'batch_size': 96, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:06,845] Trial 96 finished with value: 0.5692570209503174 and parameters: {'batch_size': 95, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:06,903] Trial 99 finished with value: 0.5195756554603577 and parameters: {'batch_size': 75, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:06,905] Trial 98 finished with value: 0.5490791201591492 and parameters: {'batch_size': 95, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:06,920] Trial 97 finished with value: 0.5477347373962402 and parameters: {'batch_size': 95, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:26,276] Trial 102 finished with value: 0.49151331186294556 and parameters: {'batch_size': 59, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:26,285] Trial 100 finished with value: 0.5133727192878723 and parameters: {'batch_size': 84, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:26,369] Trial 103 finished with value: 0.5304567813873291 and parameters: {'batch_size': 84, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:26,808] Trial 101 finished with value: 0.577867865562439 and parameters: {'batch_size': 23, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:42,355] Trial 105 finished with value: 0.5295482277870178 and parameters: {'batch_size': 98, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:42,472] Trial 104 finished with value: 0.4543357491493225 and parameters: {'batch_size': 33, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:42,480] Trial 106 finished with value: 0.47976550459861755 and parameters: {'batch_size': 91, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:23:42,487] Trial 107 finished with value: 0.4694034457206726 and parameters: {'batch_size': 34, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:01,418] Trial 108 finished with value: 0.5159963965415955 and parameters: {'batch_size': 33, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:01,437] Trial 111 finished with value: 0.5347871780395508 and parameters: {'batch_size': 38, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:01,871] Trial 110 finished with value: 0.561519980430603 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:02,327] Trial 109 finished with value: 0.5290472507476807 and parameters: {'batch_size': 34, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:15,868] Trial 114 finished with value: 0.4861186444759369 and parameters: {'batch_size': 33, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:16,040] Trial 112 finished with value: 0.6218124628067017 and parameters: {'batch_size': 33, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:16,046] Trial 113 finished with value: 0.4757813811302185 and parameters: {'batch_size': 34, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:16,321] Trial 115 finished with value: 0.5195261240005493 and parameters: {'batch_size': 34, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:35,452] Trial 116 finished with value: 0.5275002121925354 and parameters: {'batch_size': 40, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:35,575] Trial 118 finished with value: 0.5158495306968689 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:35,932] Trial 117 finished with value: 0.5653917789459229 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:36,890] Trial 119 finished with value: 0.5210202932357788 and parameters: {'batch_size': 42, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:48,627] Trial 123 finished with value: 0.5022672414779663 and parameters: {'batch_size': 29, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:48,635] Trial 121 finished with value: 0.5072609782218933 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:48,656] Trial 120 finished with value: 0.5626788139343262 and parameters: {'batch_size': 30, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:24:48,769] Trial 122 finished with value: 0.49511972069740295 and parameters: {'batch_size': 29, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:09,082] Trial 126 finished with value: 0.4857669770717621 and parameters: {'batch_size': 47, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:09,368] Trial 124 finished with value: 0.47683578729629517 and parameters: {'batch_size': 45, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:09,453] Trial 125 finished with value: 0.5855314135551453 and parameters: {'batch_size': 47, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:09,482] Trial 127 finished with value: 0.5388205647468567 and parameters: {'batch_size': 17, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:25,164] Trial 131 finished with value: 0.6068140864372253 and parameters: {'batch_size': 38, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:25,212] Trial 130 finished with value: 0.5184658765792847 and parameters: {'batch_size': 38, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:25,262] Trial 128 finished with value: 0.5416864156723022 and parameters: {'batch_size': 91, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:25,292] Trial 129 finished with value: 0.5522774457931519 and parameters: {'batch_size': 38, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:38,032] Trial 134 finished with value: 0.4604419767856598 and parameters: {'batch_size': 25, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:38,064] Trial 133 finished with value: 0.5295227766036987 and parameters: {'batch_size': 91, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:38,141] Trial 135 finished with value: 0.46299347281455994 and parameters: {'batch_size': 98, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:38,318] Trial 132 finished with value: 0.5043026208877563 and parameters: {'batch_size': 92, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:55,823] Trial 138 finished with value: 0.5082297325134277 and parameters: {'batch_size': 24, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:56,154] Trial 139 finished with value: 0.5261423587799072 and parameters: {'batch_size': 26, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:56,200] Trial 137 finished with value: 0.5447513461112976 and parameters: {'batch_size': 6, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:25:56,319] Trial 136 finished with value: 0.5455042123794556 and parameters: {'batch_size': 25, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:15,433] Trial 143 finished with value: 0.4860110878944397 and parameters: {'batch_size': 98, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:15,701] Trial 140 finished with value: 0.5036947727203369 and parameters: {'batch_size': 25, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:15,742] Trial 141 finished with value: 0.5121865272521973 and parameters: {'batch_size': 41, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:15,790] Trial 142 finished with value: 0.505050539970398 and parameters: {'batch_size': 98, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:34,644] Trial 146 finished with value: 0.49495798349380493 and parameters: {'batch_size': 99, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:34,740] Trial 147 finished with value: 0.5497980713844299 and parameters: {'batch_size': 88, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:34,741] Trial 144 finished with value: 0.5314474105834961 and parameters: {'batch_size': 31, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:35,544] Trial 145 finished with value: 0.5491929054260254 and parameters: {'batch_size': 88, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:54,448] Trial 150 finished with value: 0.514433741569519 and parameters: {'batch_size': 14, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:54,476] Trial 149 finished with value: 0.5327135324478149 and parameters: {'batch_size': 82, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:54,867] Trial 151 finished with value: 0.5778383612632751 and parameters: {'batch_size': 35, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:26:54,932] Trial 148 finished with value: 0.5008153915405273 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:13,649] Trial 154 finished with value: 0.5379410982131958 and parameters: {'batch_size': 28, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:13,760] Trial 153 finished with value: 0.4974619448184967 and parameters: {'batch_size': 36, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:14,142] Trial 152 finished with value: 0.5215822458267212 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:14,199] Trial 155 finished with value: 0.48931798338890076 and parameters: {'batch_size': 31, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:33,364] Trial 156 finished with value: 0.4868107736110687 and parameters: {'batch_size': 70, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:34,462] Trial 159 finished with value: 0.5146859884262085 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:34,506] Trial 157 finished with value: 0.5014588832855225 and parameters: {'batch_size': 31, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:34,509] Trial 158 finished with value: 0.5023144483566284 and parameters: {'batch_size': 71, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:52,844] Trial 160 finished with value: 0.5701083540916443 and parameters: {'batch_size': 43, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:54,627] Trial 162 finished with value: 0.5178627967834473 and parameters: {'batch_size': 40, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:54,648] Trial 163 finished with value: 0.48780694603919983 and parameters: {'batch_size': 76, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:27:54,670] Trial 161 finished with value: 0.528549313545227 and parameters: {'batch_size': 34, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:12,832] Trial 164 finished with value: 0.5283627510070801 and parameters: {'batch_size': 44, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:13,869] Trial 165 finished with value: 0.5243580937385559 and parameters: {'batch_size': 54, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:14,343] Trial 166 finished with value: 0.4907310903072357 and parameters: {'batch_size': 45, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:14,395] Trial 167 finished with value: 0.5622753500938416 and parameters: {'batch_size': 53, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:33,537] Trial 168 finished with value: 0.5820235013961792 and parameters: {'batch_size': 40, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:34,119] Trial 169 finished with value: 0.47089067101478577 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:34,634] Trial 171 finished with value: 0.5285822153091431 and parameters: {'batch_size': 48, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:36,019] Trial 170 finished with value: 0.5318565964698792 and parameters: {'batch_size': 48, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:53,760] Trial 172 finished with value: 0.5759711861610413 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:54,229] Trial 173 finished with value: 0.5597448945045471 and parameters: {'batch_size': 96, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:54,381] Trial 174 finished with value: 0.5148175954818726 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:28:55,692] Trial 175 finished with value: 0.51707524061203 and parameters: {'batch_size': 35, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:13,718] Trial 176 finished with value: 0.5390433073043823 and parameters: {'batch_size': 94, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:14,327] Trial 177 finished with value: 0.5315285921096802 and parameters: {'batch_size': 93, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:14,790] Trial 178 finished with value: 0.519746720790863 and parameters: {'batch_size': 93, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:16,083] Trial 179 finished with value: 0.5046969652175903 and parameters: {'batch_size': 93, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:33,134] Trial 180 finished with value: 0.562038779258728 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:33,537] Trial 181 finished with value: 0.527306854724884 and parameters: {'batch_size': 23, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:34,443] Trial 182 finished with value: 0.4859278202056885 and parameters: {'batch_size': 23, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:35,330] Trial 183 finished with value: 0.5201733708381653 and parameters: {'batch_size': 32, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:52,791] Trial 185 finished with value: 0.5056208372116089 and parameters: {'batch_size': 80, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:53,628] Trial 184 finished with value: 0.5581446290016174 and parameters: {'batch_size': 32, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:54,161] Trial 186 finished with value: 0.5182573199272156 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:29:55,052] Trial 187 finished with value: 0.49545735120773315 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:12,168] Trial 188 finished with value: 0.48832812905311584 and parameters: {'batch_size': 84, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:13,099] Trial 189 finished with value: 0.4626503586769104 and parameters: {'batch_size': 86, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:13,289] Trial 190 finished with value: 0.4701520800590515 and parameters: {'batch_size': 18, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:14,426] Trial 191 finished with value: 0.5266930460929871 and parameters: {'batch_size': 83, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:32,310] Trial 192 finished with value: 0.49622488021850586 and parameters: {'batch_size': 37, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:32,763] Trial 193 finished with value: 0.5111756920814514 and parameters: {'batch_size': 37, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:33,266] Trial 195 finished with value: 0.4457319378852844 and parameters: {'batch_size': 14, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:34,164] Trial 194 finished with value: 0.5068105459213257 and parameters: {'batch_size': 14, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:51,648] Trial 196 finished with value: 0.5240409970283508 and parameters: {'batch_size': 16, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:52,164] Trial 197 finished with value: 0.5017036199569702 and parameters: {'batch_size': 15, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:52,288] Trial 198 finished with value: 0.5609983801841736 and parameters: {'batch_size': 13, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:30:53,171] Trial 199 finished with value: 0.5506681799888611 and parameters: {'batch_size': 17, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:10,948] Trial 200 finished with value: 0.5525861978530884 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:11,373] Trial 202 finished with value: 0.5031243562698364 and parameters: {'batch_size': 18, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:11,411] Trial 201 finished with value: 0.4618286192417145 and parameters: {'batch_size': 12, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:12,706] Trial 203 finished with value: 0.5627838373184204 and parameters: {'batch_size': 9, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:30,689] Trial 204 finished with value: 0.5097069144248962 and parameters: {'batch_size': 11, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:31,074] Trial 205 finished with value: 0.5205076336860657 and parameters: {'batch_size': 10, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:31,986] Trial 207 finished with value: 0.5067445039749146 and parameters: {'batch_size': 12, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:32,460] Trial 206 finished with value: 0.5002195239067078 and parameters: {'batch_size': 11, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:50,279] Trial 209 finished with value: 0.5086316466331482 and parameters: {'batch_size': 20, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:50,617] Trial 208 finished with value: 0.48553523421287537 and parameters: {'batch_size': 9, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:51,338] Trial 210 finished with value: 0.5088384747505188 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:31:51,645] Trial 211 finished with value: 0.4787082076072693 and parameters: {'batch_size': 7, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:10,235] Trial 213 finished with value: 0.4887665808200836 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:10,582] Trial 212 finished with value: 0.5166882276535034 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:11,476] Trial 215 finished with value: 0.5511218905448914 and parameters: {'batch_size': 63, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:11,486] Trial 214 finished with value: 0.5284901261329651 and parameters: {'batch_size': 57, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:29,410] Trial 216 finished with value: 0.4862551689147949 and parameters: {'batch_size': 39, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:29,861] Trial 217 finished with value: 0.589469313621521 and parameters: {'batch_size': 89, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:31,219] Trial 218 finished with value: 0.5293298363685608 and parameters: {'batch_size': 18, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:31,231] Trial 219 finished with value: 0.5504220724105835 and parameters: {'batch_size': 45, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:49,401] Trial 220 finished with value: 0.5139113664627075 and parameters: {'batch_size': 89, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:50,238] Trial 221 finished with value: 0.4773987829685211 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:50,583] Trial 223 finished with value: 0.5001729130744934 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:32:50,591] Trial 222 finished with value: 0.5218895673751831 and parameters: {'batch_size': 45, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:08,124] Trial 224 finished with value: 0.5645706653594971 and parameters: {'batch_size': 87, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:09,472] Trial 225 finished with value: 0.5426689982414246 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:09,534] Trial 226 finished with value: 0.5196987986564636 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:09,580] Trial 227 finished with value: 0.5007689595222473 and parameters: {'batch_size': 41, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:28,084] Trial 228 finished with value: 0.4955013692378998 and parameters: {'batch_size': 41, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:29,062] Trial 230 finished with value: 0.4889008104801178 and parameters: {'batch_size': 74, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:29,370] Trial 231 finished with value: 0.5274104475975037 and parameters: {'batch_size': 15, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:29,404] Trial 229 finished with value: 0.5187384486198425 and parameters: {'batch_size': 97, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:47,154] Trial 232 finished with value: 0.49023038148880005 and parameters: {'batch_size': 26, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:48,121] Trial 234 finished with value: 0.5225495100021362 and parameters: {'batch_size': 82, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:48,403] Trial 235 finished with value: 0.5251279473304749 and parameters: {'batch_size': 27, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:33:48,499] Trial 233 finished with value: 0.5121446847915649 and parameters: {'batch_size': 39, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:07,440] Trial 236 finished with value: 0.5188892483711243 and parameters: {'batch_size': 84, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:07,600] Trial 237 finished with value: 0.5045322179794312 and parameters: {'batch_size': 78, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:08,353] Trial 239 finished with value: 0.5344282984733582 and parameters: {'batch_size': 33, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:08,475] Trial 238 finished with value: 0.4876117706298828 and parameters: {'batch_size': 34, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:27,360] Trial 241 finished with value: 0.5040766596794128 and parameters: {'batch_size': 35, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:27,489] Trial 240 finished with value: 0.565955400466919 and parameters: {'batch_size': 86, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:28,061] Trial 243 finished with value: 0.5057257413864136 and parameters: {'batch_size': 30, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:28,067] Trial 242 finished with value: 0.44389039278030396 and parameters: {'batch_size': 30, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:47,682] Trial 244 finished with value: 0.5189154148101807 and parameters: {'batch_size': 13, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:47,851] Trial 245 finished with value: 0.4862592816352844 and parameters: {'batch_size': 30, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:48,225] Trial 246 finished with value: 0.5883842706680298 and parameters: {'batch_size': 32, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:34:48,652] Trial 247 finished with value: 0.48716068267822266 and parameters: {'batch_size': 30, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:07,283] Trial 248 finished with value: 0.46146076917648315 and parameters: {'batch_size': 30, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:07,742] Trial 249 finished with value: 0.49055424332618713 and parameters: {'batch_size': 44, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:07,754] Trial 250 finished with value: 0.5191999673843384 and parameters: {'batch_size': 44, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:08,231] Trial 251 finished with value: 0.5408469438552856 and parameters: {'batch_size': 44, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:27,072] Trial 253 finished with value: 0.5437461733818054 and parameters: {'batch_size': 29, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:27,524] Trial 252 finished with value: 0.4555836617946625 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:28,799] Trial 255 finished with value: 0.5187297463417053 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:29,123] Trial 254 finished with value: 0.5010836720466614 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:47,826] Trial 256 finished with value: 0.518283486366272 and parameters: {'batch_size': 23, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:47,847] Trial 257 finished with value: 0.4808916449546814 and parameters: {'batch_size': 25, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:48,738] Trial 258 finished with value: 0.5007774233818054 and parameters: {'batch_size': 25, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:35:49,249] Trial 259 finished with value: 0.479788213968277 and parameters: {'batch_size': 25, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:07,512] Trial 261 finished with value: 0.49187541007995605 and parameters: {'batch_size': 32, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:07,674] Trial 260 finished with value: 0.5503126978874207 and parameters: {'batch_size': 27, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:08,547] Trial 262 finished with value: 0.4877100884914398 and parameters: {'batch_size': 31, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:08,986] Trial 263 finished with value: 0.5506095886230469 and parameters: {'batch_size': 21, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:27,635] Trial 265 finished with value: 0.4740053713321686 and parameters: {'batch_size': 98, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:28,068] Trial 264 finished with value: 0.45143207907676697 and parameters: {'batch_size': 21, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:28,115] Trial 266 finished with value: 0.47399917244911194 and parameters: {'batch_size': 22, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:29,455] Trial 267 finished with value: 0.49249324202537537 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:46,416] Trial 268 finished with value: 0.4758756756782532 and parameters: {'batch_size': 68, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:47,228] Trial 269 finished with value: 0.4869113564491272 and parameters: {'batch_size': 19, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:47,325] Trial 270 finished with value: 0.501675546169281 and parameters: {'batch_size': 22, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:36:49,000] Trial 271 finished with value: 0.5501865148544312 and parameters: {'batch_size': 23, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:05,789] Trial 272 finished with value: 0.5276662111282349 and parameters: {'batch_size': 22, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:07,070] Trial 273 finished with value: 0.5398856401443481 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:07,634] Trial 274 finished with value: 0.5505387783050537 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:08,577] Trial 275 finished with value: 0.5488094091415405 and parameters: {'batch_size': 19, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:24,738] Trial 276 finished with value: 0.5362070202827454 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:26,104] Trial 277 finished with value: 0.5206641554832458 and parameters: {'batch_size': 98, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:26,827] Trial 278 finished with value: 0.48881658911705017 and parameters: {'batch_size': 98, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:27,799] Trial 279 finished with value: 0.5200098156929016 and parameters: {'batch_size': 98, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:44,269] Trial 280 finished with value: 0.5378054976463318 and parameters: {'batch_size': 98, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:45,732] Trial 281 finished with value: 0.5023245811462402 and parameters: {'batch_size': 26, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:47,012] Trial 282 finished with value: 0.48922014236450195 and parameters: {'batch_size': 24, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:37:47,529] Trial 283 finished with value: 0.5231548547744751 and parameters: {'batch_size': 26, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:03,717] Trial 284 finished with value: 0.5426580905914307 and parameters: {'batch_size': 21, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:04,850] Trial 285 finished with value: 0.5048985481262207 and parameters: {'batch_size': 21, 'n_epochs': 50}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:06,241] Trial 286 finished with value: 0.5021718740463257 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:06,611] Trial 287 finished with value: 0.5073276162147522 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:23,498] Trial 288 finished with value: 0.5004807710647583 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:24,603] Trial 289 finished with value: 0.5193501710891724 and parameters: {'batch_size': 28, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:25,451] Trial 290 finished with value: 0.5197125673294067 and parameters: {'batch_size': 30, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:27,131] Trial 291 finished with value: 0.49311381578445435 and parameters: {'batch_size': 28, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:42,742] Trial 292 finished with value: 0.5729236602783203 and parameters: {'batch_size': 31, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:44,041] Trial 293 finished with value: 0.47442084550857544 and parameters: {'batch_size': 15, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:44,520] Trial 294 finished with value: 0.54139244556427 and parameters: {'batch_size': 90, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:38:45,881] Trial 295 finished with value: 0.4742720127105713 and parameters: {'batch_size': 13, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:02,136] Trial 296 finished with value: 0.47224417328834534 and parameters: {'batch_size': 90, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:03,428] Trial 297 finished with value: 0.48363685607910156 and parameters: {'batch_size': 12, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:03,873] Trial 298 finished with value: 0.504816472530365 and parameters: {'batch_size': 12, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:06,537] Trial 299 finished with value: 0.4952707886695862 and parameters: {'batch_size': 13, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:21,263] Trial 300 finished with value: 0.479077011346817 and parameters: {'batch_size': 91, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:22,345] Trial 301 finished with value: 0.4749569296836853 and parameters: {'batch_size': 92, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:22,653] Trial 302 finished with value: 0.5249071717262268 and parameters: {'batch_size': 24, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:25,778] Trial 303 finished with value: 0.497105211019516 and parameters: {'batch_size': 24, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:40,458] Trial 304 finished with value: 0.468287855386734 and parameters: {'batch_size': 24, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:41,119] Trial 305 finished with value: 0.5293434858322144 and parameters: {'batch_size': 23, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:41,511] Trial 306 finished with value: 0.5215959548950195 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:39:45,062] Trial 307 finished with value: 0.5094396471977234 and parameters: {'batch_size': 36, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:00,218] Trial 309 finished with value: 0.48130521178245544 and parameters: {'batch_size': 19, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:01,135] Trial 308 finished with value: 0.544108510017395 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:01,209] Trial 310 finished with value: 0.610135018825531 and parameters: {'batch_size': 19, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:03,865] Trial 311 finished with value: 0.551338791847229 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:19,222] Trial 312 finished with value: 0.5014687776565552 and parameters: {'batch_size': 26, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:20,584] Trial 313 finished with value: 0.4966621994972229 and parameters: {'batch_size': 29, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:20,733] Trial 314 finished with value: 0.5020628571510315 and parameters: {'batch_size': 29, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:23,150] Trial 315 finished with value: 0.5581299662590027 and parameters: {'batch_size': 29, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:38,538] Trial 316 finished with value: 0.5445773601531982 and parameters: {'batch_size': 29, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:40,280] Trial 317 finished with value: 0.5450339317321777 and parameters: {'batch_size': 32, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:41,168] Trial 318 finished with value: 0.5524824261665344 and parameters: {'batch_size': 33, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:42,149] Trial 319 finished with value: 0.47177988290786743 and parameters: {'batch_size': 22, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:58,064] Trial 320 finished with value: 0.5444153547286987 and parameters: {'batch_size': 33, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:40:59,523] Trial 321 finished with value: 0.4848056733608246 and parameters: {'batch_size': 24, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:00,508] Trial 322 finished with value: 0.49702802300453186 and parameters: {'batch_size': 22, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:01,381] Trial 323 finished with value: 0.4791908264160156 and parameters: {'batch_size': 22, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:18,548] Trial 324 finished with value: 0.4795560836791992 and parameters: {'batch_size': 22, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:20,524] Trial 325 finished with value: 0.5701809525489807 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:21,993] Trial 326 finished with value: 0.5300535559654236 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:22,007] Trial 327 finished with value: 0.5180169343948364 and parameters: {'batch_size': 26, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:38,323] Trial 328 finished with value: 0.5385854840278625 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:40,433] Trial 329 finished with value: 0.568768322467804 and parameters: {'batch_size': 25, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:41,643] Trial 331 finished with value: 0.5877790451049805 and parameters: {'batch_size': 16, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:41,982] Trial 330 finished with value: 0.5606303811073303 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:41:59,961] Trial 332 finished with value: 0.4796232283115387 and parameters: {'batch_size': 16, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:03,041] Trial 333 finished with value: 0.5005330443382263 and parameters: {'batch_size': 16, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:03,325] Trial 334 finished with value: 0.5264327526092529 and parameters: {'batch_size': 27, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:03,551] Trial 335 finished with value: 0.5752449035644531 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:20,604] Trial 336 finished with value: 0.526084303855896 and parameters: {'batch_size': 35, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:22,752] Trial 337 finished with value: 0.5624298453330994 and parameters: {'batch_size': 38, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:24,074] Trial 338 finished with value: 0.553162693977356 and parameters: {'batch_size': 6, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:24,099] Trial 339 finished with value: 0.5204086899757385 and parameters: {'batch_size': 38, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:40,190] Trial 340 finished with value: 0.6066567897796631 and parameters: {'batch_size': 24, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:42,635] Trial 341 finished with value: 0.47862738370895386 and parameters: {'batch_size': 24, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:44,172] Trial 343 finished with value: 0.49243029952049255 and parameters: {'batch_size': 14, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:42:44,636] Trial 342 finished with value: 0.47003889083862305 and parameters: {'batch_size': 23, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:00,568] Trial 344 finished with value: 0.5625410079956055 and parameters: {'batch_size': 31, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:03,819] Trial 345 finished with value: 0.5371049046516418 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:03,822] Trial 346 finished with value: 0.4888070225715637 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:04,191] Trial 347 finished with value: 0.5762099027633667 and parameters: {'batch_size': 27, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:19,911] Trial 348 finished with value: 0.5803916454315186 and parameters: {'batch_size': 28, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:22,770] Trial 349 finished with value: 0.5553117394447327 and parameters: {'batch_size': 27, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:23,206] Trial 350 finished with value: 0.4849385619163513 and parameters: {'batch_size': 27, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:23,218] Trial 351 finished with value: 0.5097953081130981 and parameters: {'batch_size': 8, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:39,404] Trial 352 finished with value: 0.5543509721755981 and parameters: {'batch_size': 34, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:42,286] Trial 353 finished with value: 0.48794570565223694 and parameters: {'batch_size': 34, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:42,737] Trial 355 finished with value: 0.5024104118347168 and parameters: {'batch_size': 35, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:43,476] Trial 354 finished with value: 0.513849139213562 and parameters: {'batch_size': 35, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:43:58,616] Trial 356 finished with value: 0.47743290662765503 and parameters: {'batch_size': 19, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:01,269] Trial 357 finished with value: 0.514382004737854 and parameters: {'batch_size': 88, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:01,855] Trial 358 finished with value: 0.541405200958252 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:02,334] Trial 359 finished with value: 0.4694004952907562 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:18,007] Trial 360 finished with value: 0.5631438493728638 and parameters: {'batch_size': 47, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:21,219] Trial 361 finished with value: 0.5177196264266968 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:21,780] Trial 363 finished with value: 0.5002210736274719 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:22,107] Trial 362 finished with value: 0.5556697249412537 and parameters: {'batch_size': 47, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:37,079] Trial 364 finished with value: 0.506619930267334 and parameters: {'batch_size': 23, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:40,259] Trial 365 finished with value: 0.5734478831291199 and parameters: {'batch_size': 23, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:40,668] Trial 366 finished with value: 0.5418880581855774 and parameters: {'batch_size': 23, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:41,098] Trial 367 finished with value: 0.5519399046897888 and parameters: {'batch_size': 24, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:44:56,663] Trial 368 finished with value: 0.5576635599136353 and parameters: {'batch_size': 17, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:00,269] Trial 369 finished with value: 0.48549318313598633 and parameters: {'batch_size': 50, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:00,288] Trial 370 finished with value: 0.502057671546936 and parameters: {'batch_size': 51, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:01,140] Trial 371 finished with value: 0.561345636844635 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:15,254] Trial 372 finished with value: 0.5147709846496582 and parameters: {'batch_size': 15, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:19,330] Trial 373 finished with value: 0.5597547292709351 and parameters: {'batch_size': 10, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:19,719] Trial 374 finished with value: 0.4734805226325989 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:20,374] Trial 375 finished with value: 0.5319303274154663 and parameters: {'batch_size': 20, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:34,206] Trial 376 finished with value: 0.4863862097263336 and parameters: {'batch_size': 56, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:38,214] Trial 377 finished with value: 0.49892091751098633 and parameters: {'batch_size': 30, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:38,934] Trial 378 finished with value: 0.5199458599090576 and parameters: {'batch_size': 29, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:40,436] Trial 379 finished with value: 0.4884510934352875 and parameters: {'batch_size': 29, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:53,102] Trial 380 finished with value: 0.5331457853317261 and parameters: {'batch_size': 21, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:57,993] Trial 381 finished with value: 0.4745132029056549 and parameters: {'batch_size': 60, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:58,122] Trial 382 finished with value: 0.5882768034934998 and parameters: {'batch_size': 21, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:45:59,801] Trial 383 finished with value: 0.4800724387168884 and parameters: {'batch_size': 25, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:11,971] Trial 384 finished with value: 0.5970281958580017 and parameters: {'batch_size': 25, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:17,339] Trial 385 finished with value: 0.5083833932876587 and parameters: {'batch_size': 96, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:17,830] Trial 386 finished with value: 0.5444313883781433 and parameters: {'batch_size': 25, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:19,117] Trial 387 finished with value: 0.47142499685287476 and parameters: {'batch_size': 96, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:33,118] Trial 388 finished with value: 0.4692153036594391 and parameters: {'batch_size': 17, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:36,394] Trial 389 finished with value: 0.548019289970398 and parameters: {'batch_size': 26, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:37,116] Trial 390 finished with value: 0.5120291113853455 and parameters: {'batch_size': 16, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:37,777] Trial 391 finished with value: 0.4730370044708252 and parameters: {'batch_size': 96, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:52,120] Trial 392 finished with value: 0.5421581268310547 and parameters: {'batch_size': 17, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:55,705] Trial 393 finished with value: 0.5373586416244507 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:56,586] Trial 394 finished with value: 0.5195258259773254 and parameters: {'batch_size': 17, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:46:56,619] Trial 395 finished with value: 0.5254815220832825 and parameters: {'batch_size': 99, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:11,454] Trial 396 finished with value: 0.5140550136566162 and parameters: {'batch_size': 100, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:15,627] Trial 397 finished with value: 0.5910347700119019 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:15,707] Trial 398 finished with value: 0.5712164640426636 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:16,093] Trial 399 finished with value: 0.522061824798584 and parameters: {'batch_size': 14, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:30,815] Trial 400 finished with value: 0.49527794122695923 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:34,549] Trial 401 finished with value: 0.5229896306991577 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:35,256] Trial 403 finished with value: 0.48684415221214294 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:35,291] Trial 402 finished with value: 0.5362198948860168 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:49,527] Trial 404 finished with value: 0.4818088412284851 and parameters: {'batch_size': 11, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:54,096] Trial 405 finished with value: 0.5026091933250427 and parameters: {'batch_size': 12, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:54,580] Trial 406 finished with value: 0.5041660070419312 and parameters: {'batch_size': 94, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:47:55,481] Trial 407 finished with value: 0.5417521595954895 and parameters: {'batch_size': 11, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:08,960] Trial 408 finished with value: 0.4659588932991028 and parameters: {'batch_size': 97, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:13,739] Trial 409 finished with value: 0.48019105195999146 and parameters: {'batch_size': 94, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:14,340] Trial 410 finished with value: 0.46940094232559204 and parameters: {'batch_size': 96, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:15,061] Trial 411 finished with value: 0.5322706699371338 and parameters: {'batch_size': 32, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:29,756] Trial 412 finished with value: 0.5059064030647278 and parameters: {'batch_size': 97, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:33,748] Trial 413 finished with value: 0.4648120403289795 and parameters: {'batch_size': 96, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:35,201] Trial 414 finished with value: 0.5509130954742432 and parameters: {'batch_size': 98, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:35,208] Trial 415 finished with value: 0.5005701780319214 and parameters: {'batch_size': 99, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:49,340] Trial 416 finished with value: 0.5380767583847046 and parameters: {'batch_size': 98, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:53,672] Trial 417 finished with value: 0.5273627638816833 and parameters: {'batch_size': 95, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:54,722] Trial 419 finished with value: 0.48017558455467224 and parameters: {'batch_size': 93, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:48:54,731] Trial 418 finished with value: 0.5458217263221741 and parameters: {'batch_size': 93, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:09,261] Trial 420 finished with value: 0.45884156227111816 and parameters: {'batch_size': 17, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:14,163] Trial 421 finished with value: 0.5154424905776978 and parameters: {'batch_size': 96, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:14,587] Trial 423 finished with value: 0.5318746566772461 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:16,488] Trial 422 finished with value: 0.5131175518035889 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:29,232] Trial 424 finished with value: 0.560625433921814 and parameters: {'batch_size': 95, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:33,998] Trial 425 finished with value: 0.5251275897026062 and parameters: {'batch_size': 16, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:34,449] Trial 426 finished with value: 0.5308502912521362 and parameters: {'batch_size': 31, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:35,845] Trial 427 finished with value: 0.49183201789855957 and parameters: {'batch_size': 33, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:49,066] Trial 428 finished with value: 0.4721307158470154 and parameters: {'batch_size': 31, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:53,992] Trial 429 finished with value: 0.5087786912918091 and parameters: {'batch_size': 32, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:54,493] Trial 430 finished with value: 0.457436740398407 and parameters: {'batch_size': 33, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:49:56,010] Trial 431 finished with value: 0.48412972688674927 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:08,651] Trial 432 finished with value: 0.5135191679000854 and parameters: {'batch_size': 29, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:13,640] Trial 433 finished with value: 0.5494176149368286 and parameters: {'batch_size': 15, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:14,950] Trial 434 finished with value: 0.48564663529396057 and parameters: {'batch_size': 36, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:15,393] Trial 435 finished with value: 0.47470125555992126 and parameters: {'batch_size': 37, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:27,874] Trial 436 finished with value: 0.5031307935714722 and parameters: {'batch_size': 36, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:32,972] Trial 437 finished with value: 0.5585656762123108 and parameters: {'batch_size': 37, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:34,360] Trial 439 finished with value: 0.5587238073348999 and parameters: {'batch_size': 32, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:34,405] Trial 438 finished with value: 0.49107351899147034 and parameters: {'batch_size': 34, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:47,537] Trial 440 finished with value: 0.5086001753807068 and parameters: {'batch_size': 91, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:53,025] Trial 441 finished with value: 0.48592454195022583 and parameters: {'batch_size': 34, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:54,611] Trial 442 finished with value: 0.5453009009361267 and parameters: {'batch_size': 30, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:50:54,765] Trial 443 finished with value: 0.5259426236152649 and parameters: {'batch_size': 97, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:06,446] Trial 444 finished with value: 0.4869484305381775 and parameters: {'batch_size': 30, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:12,486] Trial 445 finished with value: 0.5041104555130005 and parameters: {'batch_size': 97, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:13,922] Trial 447 finished with value: 0.474412202835083 and parameters: {'batch_size': 40, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:13,984] Trial 446 finished with value: 0.5817098617553711 and parameters: {'batch_size': 97, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:26,686] Trial 448 finished with value: 0.6050066947937012 and parameters: {'batch_size': 100, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:32,220] Trial 449 finished with value: 0.5036832094192505 and parameters: {'batch_size': 12, 'n_epochs': 200}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:34,078] Trial 451 finished with value: 0.4684886932373047 and parameters: {'batch_size': 33, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:34,104] Trial 450 finished with value: 0.532116711139679 and parameters: {'batch_size': 100, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:46,226] Trial 452 finished with value: 0.46315303444862366 and parameters: {'batch_size': 15, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:51,619] Trial 453 finished with value: 0.5302743315696716 and parameters: {'batch_size': 33, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:53,607] Trial 454 finished with value: 0.4795239567756653 and parameters: {'batch_size': 53, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:51:53,986] Trial 455 finished with value: 0.476126492023468 and parameters: {'batch_size': 15, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:05,254] Trial 456 finished with value: 0.4580715298652649 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:11,055] Trial 457 finished with value: 0.5125192403793335 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:13,716] Trial 458 finished with value: 0.4749816060066223 and parameters: {'batch_size': 13, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:13,771] Trial 459 finished with value: 0.5089558362960815 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:24,795] Trial 460 finished with value: 0.5232869386672974 and parameters: {'batch_size': 13, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:32,003] Trial 461 finished with value: 0.5324820876121521 and parameters: {'batch_size': 7, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:32,992] Trial 462 finished with value: 0.5542052388191223 and parameters: {'batch_size': 12, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:33,009] Trial 463 finished with value: 0.5430799126625061 and parameters: {'batch_size': 12, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:44,514] Trial 464 finished with value: 0.5459885597229004 and parameters: {'batch_size': 12, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:51,651] Trial 465 finished with value: 0.5268252491950989 and parameters: {'batch_size': 9, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:52,135] Trial 466 finished with value: 0.4818381369113922 and parameters: {'batch_size': 9, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:52:52,602] Trial 467 finished with value: 0.5289856195449829 and parameters: {'batch_size': 16, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:03,576] Trial 468 finished with value: 0.500895619392395 and parameters: {'batch_size': 9, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:11,924] Trial 469 finished with value: 0.48529618978500366 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:12,077] Trial 470 finished with value: 0.5182172060012817 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:13,069] Trial 471 finished with value: 0.520931601524353 and parameters: {'batch_size': 15, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:23,673] Trial 472 finished with value: 0.5545737743377686 and parameters: {'batch_size': 16, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:31,444] Trial 473 finished with value: 0.5297459363937378 and parameters: {'batch_size': 17, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:31,505] Trial 474 finished with value: 0.5544819831848145 and parameters: {'batch_size': 15, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:32,431] Trial 475 finished with value: 0.543802797794342 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:43,484] Trial 476 finished with value: 0.5527116060256958 and parameters: {'batch_size': 5, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:51,270] Trial 478 finished with value: 0.48611846566200256 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:52,196] Trial 477 finished with value: 0.5199516415596008 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:53:52,237] Trial 479 finished with value: 0.4810017943382263 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:02,895] Trial 480 finished with value: 0.5042141675949097 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:10,811] Trial 481 finished with value: 0.4952632486820221 and parameters: {'batch_size': 32, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:11,575] Trial 483 finished with value: 0.5110622644424438 and parameters: {'batch_size': 11, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:11,609] Trial 482 finished with value: 0.5144170522689819 and parameters: {'batch_size': 27, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:22,486] Trial 484 finished with value: 0.4699193835258484 and parameters: {'batch_size': 34, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:30,750] Trial 485 finished with value: 0.5178164839744568 and parameters: {'batch_size': 11, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:30,924] Trial 486 finished with value: 0.5341917276382446 and parameters: {'batch_size': 34, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:31,414] Trial 487 finished with value: 0.5305209159851074 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:43,094] Trial 488 finished with value: 0.5228234529495239 and parameters: {'batch_size': 16, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:50,191] Trial 490 finished with value: 0.5813038945198059 and parameters: {'batch_size': 80, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:50,220] Trial 489 finished with value: 0.47709518671035767 and parameters: {'batch_size': 80, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:54:50,990] Trial 491 finished with value: 0.5329570174217224 and parameters: {'batch_size': 62, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:02,829] Trial 492 finished with value: 0.46651268005371094 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:10,455] Trial 493 finished with value: 0.5765341520309448 and parameters: {'batch_size': 39, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:10,466] Trial 494 finished with value: 0.5041569471359253 and parameters: {'batch_size': 18, 'n_epochs': 500}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:10,896] Trial 495 finished with value: 0.500777006149292 and parameters: {'batch_size': 49, 'n_epochs': 2000}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:23,147] Trial 496 finished with value: 0.5418948531150818 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:28,291] Trial 497 finished with value: 0.47289857268333435 and parameters: {'batch_size': 28, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:28,489] Trial 498 finished with value: 0.5121833682060242 and parameters: {'batch_size': 29, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947. [I 2021-03-10 21:55:28,675] Trial 499 finished with value: 0.5026301145553589 and parameters: {'batch_size': 29, 'n_epochs': 100}. Best is trial 36 with value: 0.43997272849082947.
final_best_params = study_final.best_params
final_best_params, study_final.best_value
({'batch_size': 100, 'n_epochs': 500}, 0.43997272849082947)
hp_score_board = hp_score_board.append({'Phase':'Final Tuning - Epcoh/Batch Size',
'Best Value':study_final.best_value,
'Hyperparameters':f'{study_final.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.398870 | {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256} |
| 1 | Coarse Tuning | 0.399832 | {'drop_l1': 0.15872829858660079, 'drop_l2': 0.124404021283663, 'drop_l3': 0.1191444759931971, 'drop_l4': 0.6775070097550181, 'drop_l5': 0.32337913815894676, 'drop_l6': 0.13903817315530098, 'drop_l7': 0.39426663755264946, 'activation': 'relu', 'kernel_init': 'uniform'} |
| 2 | Fine Tuning - Optimizer | 0.388220 | {'optimizer': 'Adam', 'learning_rate': 0.00035718724976417535, 'epsilon': 3.5043808140212366e-05, 'beta_1': 0.9834335585011938, 'beta_2': 0.8583875028859446} |
| 3 | Final Tuning - Epcoh/Batch Size | 0.439973 | {'batch_size': 100, 'n_epochs': 500} |
0.433 i.e decreased by 0.1 at 500 epochs and batch size of 100. Hence we may not pick these values and use default values of 10 and 50 for batch_size and epochs respectivelystudy_final.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_batch_size | params_n_epochs | state | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.553380 | 2021-03-10 21:15:49.576119 | 2021-03-10 21:16:07.232230 | 0 days 00:00:17.656111 | 97 | 200 | COMPLETE |
| 1 | 1 | 0.503371 | 2021-03-10 21:15:49.577740 | 2021-03-10 21:16:07.250544 | 0 days 00:00:17.672804 | 97 | 100 | COMPLETE |
| 2 | 2 | 0.519673 | 2021-03-10 21:15:49.578971 | 2021-03-10 21:16:07.302271 | 0 days 00:00:17.723300 | 75 | 100 | COMPLETE |
| 3 | 3 | 0.542405 | 2021-03-10 21:15:49.579623 | 2021-03-10 21:16:07.318013 | 0 days 00:00:17.738390 | 69 | 2000 | COMPLETE |
| 4 | 4 | 0.494717 | 2021-03-10 21:16:07.236379 | 2021-03-10 21:16:24.924945 | 0 days 00:00:17.688566 | 77 | 50 | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 495 | 495 | 0.500777 | 2021-03-10 21:54:51.016386 | 2021-03-10 21:55:10.896510 | 0 days 00:00:19.880124 | 49 | 2000 | COMPLETE |
| 496 | 496 | 0.541895 | 2021-03-10 21:55:02.837805 | 2021-03-10 21:55:23.147126 | 0 days 00:00:20.309321 | 18 | 100 | COMPLETE |
| 497 | 497 | 0.472899 | 2021-03-10 21:55:10.469268 | 2021-03-10 21:55:28.291631 | 0 days 00:00:17.822363 | 28 | 100 | COMPLETE |
| 498 | 498 | 0.512183 | 2021-03-10 21:55:10.483277 | 2021-03-10 21:55:28.488715 | 0 days 00:00:18.005438 | 29 | 100 | COMPLETE |
| 499 | 499 | 0.502630 | 2021-03-10 21:55:10.926358 | 2021-03-10 21:55:28.675107 | 0 days 00:00:17.748749 | 29 | 100 | COMPLETE |
500 rows × 8 columns
optuna.visualization.plot_parallel_coordinate(study_final)
optuna.visualization.plot_optimization_history(study_final)
optuna.visualization.plot_slice(study_final)
model_best_params = { **hp_best_params, **final_best_params}
model_best_params
{'num_layers': 7,
'arch': 'D A B',
'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256],
'activation': 'relu',
'kernel_init': 'uniform',
'drop_rate_per_layer': [0.15872829858660079,
0.124404021283663,
0.1191444759931971,
0.6775070097550181,
0.32337913815894676,
0.13903817315530098,
0.39426663755264946],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7f3087be0430>,
'batch_size': 100,
'n_epochs': 500}
model_best_params['n_epochs'] = 100
model_best_params['batch_size'] = 10
model_best_params
{'num_layers': 7,
'arch': 'D A B',
'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256],
'activation': 'relu',
'kernel_init': 'uniform',
'drop_rate_per_layer': [0.15872829858660079,
0.124404021283663,
0.1191444759931971,
0.6775070097550181,
0.32337913815894676,
0.13903817315530098,
0.39426663755264946],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7f3087be0430>,
'batch_size': 10,
'n_epochs': 100}
dnn_1 = DNN_Model('regression')
m_1 = dnn_1.create_model(X_train.shape[1],1,**model_best_params)
callbacks = LivePlot(5,train_loss=loss,train_metric=metric)
# callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
display(Markdown("### Model Summary"))
print(m_final.summary())
display(Markdown(f"### Test Loss : {result[0]} Test Metric : {result[1]}"))
Model: "model_9" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_10 (InputLayer) [(None, 3)] 0 _________________________________________________________________ dense_24 (Dense) (None, 256) 1024 _________________________________________________________________ dropout_15 (Dropout) (None, 256) 0 _________________________________________________________________ activation_15 (Activation) (None, 256) 0 _________________________________________________________________ batch_normalization_15 (Batc (None, 256) 768 _________________________________________________________________ dense_25 (Dense) (None, 32) 8224 _________________________________________________________________ dropout_16 (Dropout) (None, 32) 0 _________________________________________________________________ activation_16 (Activation) (None, 32) 0 _________________________________________________________________ batch_normalization_16 (Batc (None, 32) 96 _________________________________________________________________ dense_26 (Dense) (None, 1000) 33000 _________________________________________________________________ dropout_17 (Dropout) (None, 1000) 0 _________________________________________________________________ activation_17 (Activation) (None, 1000) 0 _________________________________________________________________ batch_normalization_17 (Batc (None, 1000) 3000 _________________________________________________________________ dense_27 (Dense) (None, 1000) 1001000 _________________________________________________________________ dropout_18 (Dropout) (None, 1000) 0 _________________________________________________________________ activation_18 (Activation) (None, 1000) 0 _________________________________________________________________ batch_normalization_18 (Batc (None, 1000) 3000 _________________________________________________________________ dense_28 (Dense) (None, 10) 10010 _________________________________________________________________ dropout_19 (Dropout) (None, 10) 0 _________________________________________________________________ activation_19 (Activation) (None, 10) 0 _________________________________________________________________ batch_normalization_19 (Batc (None, 10) 30 _________________________________________________________________ dense_29 (Dense) (None, 32) 352 _________________________________________________________________ dropout_20 (Dropout) (None, 32) 0 _________________________________________________________________ activation_20 (Activation) (None, 32) 0 _________________________________________________________________ batch_normalization_20 (Batc (None, 32) 96 _________________________________________________________________ dense_30 (Dense) (None, 256) 8448 _________________________________________________________________ dropout_21 (Dropout) (None, 256) 0 _________________________________________________________________ activation_21 (Activation) (None, 256) 0 _________________________________________________________________ batch_normalization_21 (Batc (None, 256) 768 _________________________________________________________________ dense_31 (Dense) (None, 1) 257 ================================================================= Total params: 1,070,073 Trainable params: 1,064,901 Non-trainable params: 5,172 _________________________________________________________________ None
hp_score_board = hp_score_board.append({'Phase':'Model Testing - Train/Test',
'Best Value':f'{loss}: {result[0]} - {metric}: {result[1]}',
'Hyperparameters':f'{model_best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.39887 | {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32, 'neuron_l7': 256} |
| 1 | Coarse Tuning | 0.399832 | {'drop_l1': 0.15872829858660079, 'drop_l2': 0.124404021283663, 'drop_l3': 0.1191444759931971, 'drop_l4': 0.6775070097550181, 'drop_l5': 0.32337913815894676, 'drop_l6': 0.13903817315530098, 'drop_l7': 0.39426663755264946, 'activation': 'relu', 'kernel_init': 'uniform'} |
| 2 | Fine Tuning - Optimizer | 0.38822 | {'optimizer': 'Adam', 'learning_rate': 0.00035718724976417535, 'epsilon': 3.5043808140212366e-05, 'beta_1': 0.9834335585011938, 'beta_2': 0.8583875028859446} |
| 3 | Final Tuning - Epcoh/Batch Size | 0.439973 | {'batch_size': 100, 'n_epochs': 500} |
| 4 | Model Testing - Train/Test | mean_absolute_error: 0.4081677794456482 - mean_absolute_error: 0.4081677794456482 | {'num_layers': 7, 'arch': 'D A B', 'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256], 'activation': 'relu', 'kernel_init': 'uniform', 'drop_rate_per_layer': [0.15872829858660079, 0.124404021283663, 0.1191444759931971, 0.6775070097550181, 0.32337913815894676, 0.13903817315530098, 0.39426663755264946], 'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam object at 0x7f3087be0430>, 'batch_size': 10, 'n_epochs': 100} |
# opt_conf
temp_bp = model_best_params
temp_bp['opts'] = opt_conf
temp_bp
{'num_layers': 7,
'arch': 'D A B',
'neurons_per_layer': [256, 32, 1000, 1000, 10, 32, 256],
'activation': 'relu',
'kernel_init': 'uniform',
'drop_rate_per_layer': [0.15872829858660079,
0.124404021283663,
0.1191444759931971,
0.6775070097550181,
0.32337913815894676,
0.13903817315530098,
0.39426663755264946],
'opts': {'optimizer': 'Adam',
'learning_rate': 0.00035718724976417535,
'epsilon': 3.5043808140212366e-05,
'beta_1': 0.9834335585011938,
'beta_2': 0.8583875028859446},
'batch_size': 10,
'n_epochs': 100}
Model Based Feature Selection was better than the other two.A multi stage strategy was applied in tuning hyper parameters of the ANN.
Stage 1: Architecture Selection - MAE of 0.39887
Stage 2: Coarse tuning - MAE of 0.399832
Stage 3 : Fine tuning - MAE of 0.38822
Satge 4 : Final tuning - MAE of 0.439973
However, all hyperparameters prior to final tuning were retained and defaults applied to stage 4 parameters.
Streamlit an opensource is created to test the hyperparameters and to save the model. You can try the same @ http://34.70.58.49:8085/. Please make sure to upload the data file and copy paste the hyperparameters above in the respective text input area.